Collective Intelligence

23418 2

Outline

bull What is Swarm Intelligence (SI)

bull Multi-Agents System (MAS)

bull Simulate SI for Search

ndash Ant Colony Optimization (ACO)

ndash Particle Swarm Optimization (PSO)

23418 3

bull Some social systems in Nature can present an intelligent collective behavior although they are composed by simple individuals

bull The intelligent solutions to problems naturally emerge from the self-organization and communication of these individuals

bull These systems provide important techniques that can be used in the development of artificial intelligent systems

The Computational Beauty of Nature

23418 4

Examples of Collective Behavior in Nature and Society

bull Many agents (individualpart)Many agents (individualpart)

bull Local and simple interactionsLocal and simple interactions

bull New properties New properties emergeemerge bull phase transition pattern formation group movement hellipphase transition pattern formation group movement hellip

Which can be treated as Multi-Agent System

23418 2

Outline

bull What is Swarm Intelligence (SI)

bull Multi-Agents System (MAS)

bull Simulate SI for Search

ndash Ant Colony Optimization (ACO)

ndash Particle Swarm Optimization (PSO)

23418 3

bull Some social systems in Nature can present an intelligent collective behavior although they are composed by simple individuals

bull The intelligent solutions to problems naturally emerge from the self-organization and communication of these individuals

bull These systems provide important techniques that can be used in the development of artificial intelligent systems

The Computational Beauty of Nature

23418 4

Examples of Collective Behavior in Nature and Society

bull Many agents (individualpart)Many agents (individualpart)

bull Local and simple interactionsLocal and simple interactions

bull New properties New properties emergeemerge bull phase transition pattern formation group movement hellipphase transition pattern formation group movement hellip

Which can be treated as Multi-Agent System

23418 3

bull Some social systems in Nature can present an intelligent collective behavior although they are composed by simple individuals

bull The intelligent solutions to problems naturally emerge from the self-organization and communication of these individuals

bull These systems provide important techniques that can be used in the development of artificial intelligent systems

The Computational Beauty of Nature

23418 4

Examples of Collective Behavior in Nature and Society

bull Many agents (individualpart)Many agents (individualpart)

bull Local and simple interactionsLocal and simple interactions

bull New properties New properties emergeemerge bull phase transition pattern formation group movement hellipphase transition pattern formation group movement hellip

Which can be treated as Multi-Agent System

23418 4

Examples of Collective Behavior in Nature and Society

bull Many agents (individualpart)Many agents (individualpart)

bull Local and simple interactionsLocal and simple interactions

bull New properties New properties emergeemerge bull phase transition pattern formation group movement hellipphase transition pattern formation group movement hellip

Which can be treated as Multi-Agent System

23418 5

Emergencebull Goldstein ldquoThe arising of novel and

coherent structures patterns and properties during the process of self-organization in complex systems

bull Murray Gell-Mann ldquoSuperficial complexity that arises from a deep simplicityrdquo

bull Bottom-up behavior Simple agents following simple rules generate complex structuresbehaviors Agents donrsquot follow

orders from a leader

A termite cathedral A termite cathedral mound produced by a mound produced by a termite colony a classic termite colony a classic example of emergence in example of emergence in nature nature

23418 6

bull Colonies of social insects can achieve flexible reliable intelligent complex system level performance from insect elements which are stereotyped unreliable unintelligent and simple

bull Insects follow simple rules use simple local communication (scent trails sound touch) with low computational demands

bull Global structure (eg nest) reliably emerges from the unreliable actions of many

Biological motivation Insect Societies

23418 7

bull Collective systems capable of accomplishing difficult tasks in dynamic and varied environments without any external guidance or control and with no central coordination

bull Achieving a collective performance which could not normally be achieved by any individual acting alone

bull The colony as a whole is the seat of a stable and self-regulated organization of individual behavior which adapts itself very easily to the unpredictable characteristics of the environment within which it evolved

Insect Societies

23418 8

Self Organizationbull Insect societies have developed systems of collective

decision making operating without symbolic representations exploiting the physical constraints of the environment in which they evolved and using communications between individuals either directly when in contact or indirectly (stigmergy) using the environment as a channel of communication

bull Through these direct and indirect interactions the society self organizes and faced with a problem finds a solution with a complexity far greater than that of the insects of which it is composed

23418 9

Stigmergy

bull Indirect communication via interaction with environment [Gasseacute 59]ndash Sematonic [Wilson 75] stigmergy

bull action of agent directly related to problem solving and affects behavior of other agents

ndash Sign-based stigmergybull action of agent affects environment not directly

related to problem solving activity

23418 10

Flocks Herds and Schoolsbull In the late 80rsquos Craig Reynolds created a simple model of animal motion that he called Boids

bull Itrsquos generates very realistic motion for movement from three simple rules which define a boidrsquos steering behaviour

bull This model and its variations has been used to drive animations of birds insects people fish antelope etc in films (eg Batman Returns Lion King)

23418 11

Boid rulesSeparation steer to avoid crowding local flockmates

- A fundamental rule that has priority over the others- Also useful in avoiding collisions with other objects in the environment

Alignment steer towards the average heading and speed of local flockmates

- Enforces cohesion to keep the flock togetherHelps with collision avoidance too

Cohesion steer to move toward the average position of local flockmates

- Agents at edge of the herd more vulnerable to predators

- Helps to keep the flock together

23418 12

bull Swarm intelligence (SI) is an artificial intelligence technique based around the study of collective behavior in decentralized self-organized systems

bull The express lsquoSwarm Intelligencersquo was originally used by Beni Hackwood and Wang in 1989 in the context of cellular robotic systems to describe the self-organization of simple mechanical agents

bull It was later extended to include ldquoany attempt to design algorithms or distributed problem-solving devices inspired by the collective behavior of social insect colonies and other animal societiesrdquo [Bonabeau Dorigo and Theraulaz 1999]

Swarm Intelligence

23418 13

Swarm Intelligence (Contrsquod)

bull SI systems are typically made up of a population of simple agents interacting locally with one another and with their environment

bull Although there is normally no centralized control structure dictating how individual agents should behave local interactions between such agents often lead to the emergence of global behavior

bull Sometimes called lsquoCollective Intelligencersquo

23418 14

Components of SIbull Agents

ndash Interact with the world and with each other (either directly or indirectly)

bull Simple behavioursndash eg ants termites bees wasps

bull Communicationndash How agents interact with each otherndash eg pheromones of ants

Simple behaviours of individual agents

+ Communication between a group of agents

= Emergent complex behaviour of the group of agents

23418 15

Characteristics of SI

bull Distributed no central control or data source

bull Limited communication

bull No (explicit) model of the environment

bull Perception of environment (sensing)

bull Ability to react to environment changes

Is SI relevant to people

23418 16

The Web becomes a Giant Brain

Some see the Web evolving

intoa collective

brain for humankind

23418 17

What is Multi-agent Systemsbull A set of agents which interact in a common environmentbull Focus on the collaborative resolution of global

problems by a set of distributed entitiesbull Agent attempt to satisfy their own local goals as well as

the collaborative global goals bull To successfully interact they will require the ability to

cooperate coordinate and negotiate with each other much as people do

23418 18

What is MAS(Contrsquod)

bull MAS as seen from Distributed AIndash a loosely coupled network of entities that work

together to find answers to problems that are beyond the individual capabilities or knowledge of each entity

bull A more general meaningndash systems composed of autonomous components

that exhibit the following characteristicsbull each agent has incomplete capabilities to solve a problembull there is no global system controlbull data is decentralizedbull computation is asynchronous

23418 19

bull Traditionalndash Client-serverndash Low-level

messagesndash Synchronousndash Can not do the job

bull Agent breakthroughsndash Peer-to-peer

topologyndash Blackboard

coordination modelndash Encapsulated

messagingndash High-level message

protocols

Client ServerFunction(Parameters)

Return(Parameters)

Traditional Software

IntelligentAgents

IntelligentAgents

IntelligentAgents

Blackboard

MessageReply

Agents IntelligentAgentsIntelligent

Agents

IntelligentAgentsIntelligent

Agents

IntelligentAgentsIntelligent

Agents

23418 20

Communication models

bull Theoretical models Speech act theory

bull Practical modelsndash shared languages like KIF KQML DAMLndash service models like DAML-Sndash social convention protocols

23418 21

Working together

bull Benevolent Agentsndash assume agents are benevolent our best

interest is their best interest

bull Self-Interested Agentsndash Agents will be assumed to act to further their

own interests possibly at expense of othersndash Potential for conflict

23418 22

Example mechanism Contract Net Protocol (CNP)

bull Negotiation as a collaboration mechanism

bull Negotiation on how tasks should be shared

ndash A task (plan) may be decomposed in a hierarchy of subtasks (hierarchical planning)

ndash An agent may subcontract another agent to perform a (sub)task

Contract

Bid

agent agent

CNP

Task announceme

nt

23418 24

CNP (Contrsquod)

Contractor

Potential candidate agents

Task announcement (broadcast)

Contractor

Candidate Candidate

Bid

Bid

Phase 1 Task Announcement

- The contractor agent publicly announces a task

- Potential candidates evaluate the task according to their won skills and availability

Phase 2 Submission of Bids Proposals

- Agents that satisfy the requiremenst ie are able to perform the task send their bid proposal to the contractor

23418 25

CNP (Contrsquod)

Contractor

Selected candidate

Contractor

Contracted agent

Contract

Phase 3 Selection

- The selection of the best candidate is made by the contractor based on received bids and on the CVs of the candidates

Phase 4 Contract awarding

- A contract is established between the contractor and the selected candidate

- A privileged bilateral communication channel is established between the two agents

23418 26

Attributes of Multi-agent Systems

Apply MAS when some of the following features show up in a problem

bull Decentralizationbull Complex components often best described at

the knowledge levelbull Adaptive behaviorbull Complex interactionsbull Coordinationbull Emergent aggregate behaviors

23418 27

Applications of MAS

Advanced Manufacturing Management Systemsminus Agents as representatives of machines users business

processes etc

Intelligent Information Search on Internetminus Some agents may show learning capabilities (learn the

preferences of their users )

Intelligent security enforcement on Internetminus Agents are representative of sensors or IDSs

Shopping Agents in Electronic Commerceminus With search price comparison and bargaining capabilities

23418 28

Applications of MAS (Contrsquod)

Multi-agent auction in E-commerce

Distributed Surveillanceminus For information search or to look for special events informing

their users of relevant news

Distributed Signal Processingminus For problem diagnosis situation assessment etc in the

network

Distributed Problem Solvingminus Collaborative design scheduling and planning

23418 29

How to simulate SI for search

Example1 Ant --gt Ant Colony Optimization (ACO)

Example2 Bird Flocking --gt Particle Swarm Optimization (PSO)

23418 30

Part Ant Colony Optimization Ⅱ(ACO)

First proposed by M Dorigo 1992

23418 31

Natural Antsbull Individual ants are simple insects with

limited memory and capable of performing simple actions

bull However an ant colony expresses a complex collective behavior providing intelligent solutions to problems such asndash carrying large itemsndash forming bridgesndash finding the shortest routes from the nest

to a food source prioritizing food sources based on their distance and ease of access

23418 32

Natural Antsbull Moreover in a colony each ant has its prescribed task but

the ants can switch tasks if the collective needs it ndash Outside the nest ants can have 4 different tasks

bull Foraging searching for and retrieving food

bull Patrolling looking for food supply

bull Midden work Sorting the colony refuse pile

bull Nest maintenance work construction and clearing of chambers

ndash An antrsquos decision whether to perform a task depends onbull The Phisical State of the environment

ndash If part of the nest is damaged more ants do nest maintenance work to repair it

bull Social Interactions with other ants

Communication (direct or indirect) is necessary

23418 33

bull They establish indirect communication system based on the deposition of pheromone over the path they followndash An isolated ant moves at random but when it finds a pheromone

trail there is a high probability that this ant will decide to follow the trail

ndash An ant foraging for food deposits pheromone over its route When it finds a food source it returns to the nest reinforcing its trail

ndash So other ants have greater probability to start following this trail and thereby laying more pheromone on it

ndash This process works as a positive feedback loop system because the higher the intensity of the pheromone over a trail the higher the probability of an ant start traveling through it

How can the natural ants find the shortest path

23418 34

bull Since the route B is shorter the ants on this path will complete the travel more times and thereby lay more pheromone over it

bull The pheromone concentration on trail B will increase at a higher rate than on A and soon the ants on route A will choose to follow route B

bull Since most ants will no longer travel on route A and since the pheromone is volatile trail A will start evaporating

bull Only the shortest route will remain

23418 35

Problems of ASbull Ant System tends to converge quickly

ndash This means that its exploitation of the best solution found is too high it should be exploring solution space more

ndash Pheromone evaporationupdate rule (better rule may exist)

bull Led to extensions of the ant systemndash Elitist Strategy for Ant Systems (EAS)ndash Rank based Ant Systems (AKRANK)ndash MAX-MIN Ant system (MMAS)ndash Ant Colony System (ACS)

23418 36

Part Ⅲ Particle Swarm Optimization (PSO)

1048713Firstly Proposed by Kennedy and Eberhart 1995

ldquoOnce again nature has provided us with a technique for processing information that is at once elegant and versatilerdquo

23418 37

bull In PSO a ldquoswarmrdquo is defined as an apparently disorganized collection (population) of moving individuals that tend to cluster together while each individual seems to be moving in a random direction

Bird flocking is one of the best example of PSO in nature

23418 38

Modeling bird flocking

bull The synchrony of flocking behavior is thought to be a function of birdrsquos efforts to maintain an optimal distance between themselves and their neighborsndash Birds and fish adjust their physical movement to avoid

predators seek for food and mates ndash Humans tend to adjust our beliefs and attitudes to

conform with those of our social peers Humans change in abstract multidimensional space collision-free

23418 39

Modeling bird flocking (Contrsquod)

bull Definitions ndash Flock is a group of objects that exhibit the general

class of polarized (aligned) non-colliding aggregate motion

ndash Boid is a simulated bird-like object ie it exhibits this type of behavior It can be a fish bee dinosaur etc

bull Rules for flockingndash Cohesion Each boid fly towards the centroid of its

local flock mates (that is boid in its local neighborhood)ndash Separation Each boid keep a certain distance away

from local flock mates to avoid collisionsndash Alignment Each boid align its velocity vector and keep

velocity magnitude similar with that of the local flock

23418 40

bull Imagine a birdrsquos flock in an area where there is a single food source

bull A bird donrsquot know where the food is but it knows its distance to the food

bull The best strategy is to follow the bird that is closer to the food

bull Particles save and communicate the best solution they have found

From Bird to Particle

23418 41

Features of PSO

bull Population initialized by assigning random positions and velocities potential solutions are then flown through hyperspace

bull Each particle keeps track of its ldquobestrdquo (highest fitness) position in hyperspacendash This is called ldquopBestrdquo for an individual particlendash It is called ldquogBestrdquo for the best in the populationndash It is called ldquolBestrdquo for the best in a defined neighborhood

bull At each time step each particle stochastically accelerates toward its pBest and gBest (or lBest)

23418 42

Particle Swarm Optimization Process

Step1 Initialize population in hyperspace

Step2 Evaluate fitness of individual particles

Step3 Modify velocities based on previous best and global (or neighborhood) best

Step4 Terminate on some condition

Step5 Go to step 2

23418 43

How do particles fly

bull Combination of gBest and the pBest (lBest)ndash need a compromise

bull lBest can bendash Social the particles around are always the same no matter

where they are in spacendash Geographical the particles around are those whose distance is

the shortest

bull Global PSO vs Local PSOndash the global version converges quickly to a solution but it gets

more easily stuck in local minima

23418 44

Illustrating the velocity update schema of global PSO

23418 45

PSO Related issues

bull Controlling velocities (determining the best value for Vmax)ndash Usually set maximum velocity to dynamic range of variable

bull Usually set c1 and c2 to 2

bull Inertia weight influences tradeoff between global and local explorationndash Good approach is to reduce inertia weight during run (ie from

09 to 04 over 1000 generations)

bull Swarm Size and Neighborhood Size

23418 46

Advantages of PSO

bull Adaptation operates on velocitiesndash Most other methods operate on positionsndash Effect PSO has a builtin momentumndash Particles tend to hurdle past optima ndash an advantage

since the best positions are remembered anyway

bull Simple in conceptbull Easy to implementbull Computationally efficientbull Effective on a variety of problems

23418 47

Summarybull Swarm Intelligence (SI)

ndash an artificial intelligence technique based around the study of collective behavior in decentralized self-organized systems

bull Multi-Agent Systems (MAS)ndash A system that consists of a number of agents

which interact with one-anotherndash Communication Coordination

Collaboration

23418 48

bull Ant Colony Optimization (ACO)ndash Inspired by ant colony foragingndash pheromone as heuristic information

(stigmergy)ndash Iteration between ConstructAntSolutions and

UpdatePheromones

bull Particle Swarm Optimization (PSO)ndash Inspired by bird flockingndash Heuristic information results from partnersndash Particle Velocity Update

• Collective Intelligence
• Outline
• The Computational Beauty of Nature
• Examples of Collective Behavior in Nature and Society
• Emergence
• Biological motivation Insect Societies
• Insect Societies
• Self Organization
• Stigmergy
• Flocks Herds and Schools
• Boid rules
• Slide 12
• Swarm Intelligence (Contrsquod)
• Components of SI
• Characteristics of SI
• The Web becomes a Giant Brain Some see the Web evolving into a collective brain for humankind
• What is Multi-agent Systems
• What is MAS(Contrsquod)
• Slide 19
• Communication models
• Working together
• Example mechanism Contract Net Protocol (CNP)
• Slide 23
• CNP (Contrsquod)
• Slide 25
• Attributes of Multi-agent Systems
• Applications of MAS
• Applications of MAS (Contrsquod)
• How to simulate SI for search
• Part Ⅱ Ant Colony Optimization (ACO)
• Natural Ants
• Slide 32
• How can the natural ants find the shortest path
• Slide 34
• Problems of AS
• Part Ⅲ Particle Swarm Optimization (PSO)
• Slide 37
• Modeling bird flocking
• Modeling bird flocking (Contrsquod)
• From Bird to Particle
• Features of PSO
• Particle Swarm Optimization Process
• How do particles fly
• Slide 44
• PSO Related issues
• Advantages of PSO
• Summary
• Slide 48

23418 6

bull Colonies of social insects can achieve flexible reliable intelligent complex system level performance from insect elements which are stereotyped unreliable unintelligent and simple

bull Insects follow simple rules use simple local communication (scent trails sound touch) with low computational demands

bull Global structure (eg nest) reliably emerges from the unreliable actions of many

Biological motivation Insect Societies

23418 7

bull Collective systems capable of accomplishing difficult tasks in dynamic and varied environments without any external guidance or control and with no central coordination

bull Achieving a collective performance which could not normally be achieved by any individual acting alone

bull The colony as a whole is the seat of a stable and self-regulated organization of individual behavior which adapts itself very easily to the unpredictable characteristics of the environment within which it evolved

Insect Societies

23418 8

Self Organizationbull Insect societies have developed systems of collective

decision making operating without symbolic representations exploiting the physical constraints of the environment in which they evolved and using communications between individuals either directly when in contact or indirectly (stigmergy) using the environment as a channel of communication

bull Through these direct and indirect interactions the society self organizes and faced with a problem finds a solution with a complexity far greater than that of the insects of which it is composed

23418 9

Stigmergy

bull Indirect communication via interaction with environment [Gasseacute 59]ndash Sematonic [Wilson 75] stigmergy

bull action of agent directly related to problem solving and affects behavior of other agents

ndash Sign-based stigmergybull action of agent affects environment not directly

related to problem solving activity

23418 10

Flocks Herds and Schoolsbull In the late 80rsquos Craig Reynolds created a simple model of animal motion that he called Boids

bull Itrsquos generates very realistic motion for movement from three simple rules which define a boidrsquos steering behaviour

bull This model and its variations has been used to drive animations of birds insects people fish antelope etc in films (eg Batman Returns Lion King)

23418 11

Boid rulesSeparation steer to avoid crowding local flockmates

- A fundamental rule that has priority over the others- Also useful in avoiding collisions with other objects in the environment

Alignment steer towards the average heading and speed of local flockmates

- Enforces cohesion to keep the flock togetherHelps with collision avoidance too

Cohesion steer to move toward the average position of local flockmates

- Agents at edge of the herd more vulnerable to predators

- Helps to keep the flock together

23418 12

bull Swarm intelligence (SI) is an artificial intelligence technique based around the study of collective behavior in decentralized self-organized systems

bull The express lsquoSwarm Intelligencersquo was originally used by Beni Hackwood and Wang in 1989 in the context of cellular robotic systems to describe the self-organization of simple mechanical agents

bull It was later extended to include ldquoany attempt to design algorithms or distributed problem-solving devices inspired by the collective behavior of social insect colonies and other animal societiesrdquo [Bonabeau Dorigo and Theraulaz 1999]

Swarm Intelligence

23418 13

Swarm Intelligence (Contrsquod)

bull SI systems are typically made up of a population of simple agents interacting locally with one another and with their environment

bull Although there is normally no centralized control structure dictating how individual agents should behave local interactions between such agents often lead to the emergence of global behavior

bull Sometimes called lsquoCollective Intelligencersquo

23418 14

Components of SIbull Agents

ndash Interact with the world and with each other (either directly or indirectly)

bull Simple behavioursndash eg ants termites bees wasps

bull Communicationndash How agents interact with each otherndash eg pheromones of ants

Simple behaviours of individual agents

+ Communication between a group of agents

= Emergent complex behaviour of the group of agents

23418 15

Characteristics of SI

bull Distributed no central control or data source

bull Limited communication

bull No (explicit) model of the environment

bull Perception of environment (sensing)

bull Ability to react to environment changes

Is SI relevant to people

23418 16

The Web becomes a Giant Brain

Some see the Web evolving

intoa collective

brain for humankind

23418 17

What is Multi-agent Systemsbull A set of agents which interact in a common environmentbull Focus on the collaborative resolution of global

problems by a set of distributed entitiesbull Agent attempt to satisfy their own local goals as well as

the collaborative global goals bull To successfully interact they will require the ability to

cooperate coordinate and negotiate with each other much as people do

23418 18

What is MAS(Contrsquod)

bull MAS as seen from Distributed AIndash a loosely coupled network of entities that work

together to find answers to problems that are beyond the individual capabilities or knowledge of each entity

bull A more general meaningndash systems composed of autonomous components

that exhibit the following characteristicsbull each agent has incomplete capabilities to solve a problembull there is no global system controlbull data is decentralizedbull computation is asynchronous

23418 19

bull Traditionalndash Client-serverndash Low-level

messagesndash Synchronousndash Can not do the job

bull Agent breakthroughsndash Peer-to-peer

topologyndash Blackboard

coordination modelndash Encapsulated

messagingndash High-level message

protocols

Client ServerFunction(Parameters)

Return(Parameters)

Traditional Software

IntelligentAgents

IntelligentAgents

IntelligentAgents

Blackboard

MessageReply

Agents IntelligentAgentsIntelligent

Agents

IntelligentAgentsIntelligent

Agents

IntelligentAgentsIntelligent

Agents

23418 20

Communication models

bull Theoretical models Speech act theory

bull Practical modelsndash shared languages like KIF KQML DAMLndash service models like DAML-Sndash social convention protocols

23418 21

Working together

bull Benevolent Agentsndash assume agents are benevolent our best

interest is their best interest

bull Self-Interested Agentsndash Agents will be assumed to act to further their

own interests possibly at expense of othersndash Potential for conflict

23418 22

Example mechanism Contract Net Protocol (CNP)

bull Negotiation as a collaboration mechanism

bull Negotiation on how tasks should be shared

ndash A task (plan) may be decomposed in a hierarchy of subtasks (hierarchical planning)

ndash An agent may subcontract another agent to perform a (sub)task

Contract

Bid

agent agent

CNP

Task announceme

nt

23418 24

CNP (Contrsquod)

Contractor

Potential candidate agents

Task announcement (broadcast)

Contractor

Candidate Candidate

Bid

Bid

Phase 1 Task Announcement

- The contractor agent publicly announces a task

- Potential candidates evaluate the task according to their won skills and availability

Phase 2 Submission of Bids Proposals

- Agents that satisfy the requiremenst ie are able to perform the task send their bid proposal to the contractor

23418 25

CNP (Contrsquod)

Contractor

Selected candidate

Contractor

Contracted agent

Contract

Phase 3 Selection

- The selection of the best candidate is made by the contractor based on received bids and on the CVs of the candidates

Phase 4 Contract awarding

- A contract is established between the contractor and the selected candidate

- A privileged bilateral communication channel is established between the two agents

23418 26

Attributes of Multi-agent Systems

Apply MAS when some of the following features show up in a problem

bull Decentralizationbull Complex components often best described at

the knowledge levelbull Adaptive behaviorbull Complex interactionsbull Coordinationbull Emergent aggregate behaviors

23418 27

Applications of MAS

Advanced Manufacturing Management Systemsminus Agents as representatives of machines users business

processes etc

Intelligent Information Search on Internetminus Some agents may show learning capabilities (learn the

preferences of their users )

Intelligent security enforcement on Internetminus Agents are representative of sensors or IDSs

Shopping Agents in Electronic Commerceminus With search price comparison and bargaining capabilities

23418 28

Applications of MAS (Contrsquod)

Multi-agent auction in E-commerce

Distributed Surveillanceminus For information search or to look for special events informing

their users of relevant news

Distributed Signal Processingminus For problem diagnosis situation assessment etc in the

network

Distributed Problem Solvingminus Collaborative design scheduling and planning

23418 29

How to simulate SI for search

Example1 Ant --gt Ant Colony Optimization (ACO)

Example2 Bird Flocking --gt Particle Swarm Optimization (PSO)

23418 30

Part Ant Colony Optimization Ⅱ(ACO)

First proposed by M Dorigo 1992

23418 31

Natural Antsbull Individual ants are simple insects with

limited memory and capable of performing simple actions

bull However an ant colony expresses a complex collective behavior providing intelligent solutions to problems such asndash carrying large itemsndash forming bridgesndash finding the shortest routes from the nest

to a food source prioritizing food sources based on their distance and ease of access

23418 32

Natural Antsbull Moreover in a colony each ant has its prescribed task but

the ants can switch tasks if the collective needs it ndash Outside the nest ants can have 4 different tasks

bull Foraging searching for and retrieving food

bull Patrolling looking for food supply

bull Midden work Sorting the colony refuse pile

bull Nest maintenance work construction and clearing of chambers

ndash An antrsquos decision whether to perform a task depends onbull The Phisical State of the environment

ndash If part of the nest is damaged more ants do nest maintenance work to repair it

bull Social Interactions with other ants

Communication (direct or indirect) is necessary

23418 33

bull They establish indirect communication system based on the deposition of pheromone over the path they followndash An isolated ant moves at random but when it finds a pheromone

trail there is a high probability that this ant will decide to follow the trail

ndash An ant foraging for food deposits pheromone over its route When it finds a food source it returns to the nest reinforcing its trail

ndash So other ants have greater probability to start following this trail and thereby laying more pheromone on it

ndash This process works as a positive feedback loop system because the higher the intensity of the pheromone over a trail the higher the probability of an ant start traveling through it

How can the natural ants find the shortest path

23418 34

bull Since the route B is shorter the ants on this path will complete the travel more times and thereby lay more pheromone over it

bull The pheromone concentration on trail B will increase at a higher rate than on A and soon the ants on route A will choose to follow route B

bull Since most ants will no longer travel on route A and since the pheromone is volatile trail A will start evaporating

bull Only the shortest route will remain

23418 35

Problems of ASbull Ant System tends to converge quickly

ndash This means that its exploitation of the best solution found is too high it should be exploring solution space more

ndash Pheromone evaporationupdate rule (better rule may exist)

bull Led to extensions of the ant systemndash Elitist Strategy for Ant Systems (EAS)ndash Rank based Ant Systems (AKRANK)ndash MAX-MIN Ant system (MMAS)ndash Ant Colony System (ACS)

23418 36

Part Ⅲ Particle Swarm Optimization (PSO)

1048713Firstly Proposed by Kennedy and Eberhart 1995

ldquoOnce again nature has provided us with a technique for processing information that is at once elegant and versatilerdquo

23418 37

bull In PSO a ldquoswarmrdquo is defined as an apparently disorganized collection (population) of moving individuals that tend to cluster together while each individual seems to be moving in a random direction

Bird flocking is one of the best example of PSO in nature

23418 38

Modeling bird flocking

bull The synchrony of flocking behavior is thought to be a function of birdrsquos efforts to maintain an optimal distance between themselves and their neighborsndash Birds and fish adjust their physical movement to avoid

predators seek for food and mates ndash Humans tend to adjust our beliefs and attitudes to

conform with those of our social peers Humans change in abstract multidimensional space collision-free

23418 39

Modeling bird flocking (Contrsquod)

bull Definitions ndash Flock is a group of objects that exhibit the general

class of polarized (aligned) non-colliding aggregate motion

ndash Boid is a simulated bird-like object ie it exhibits this type of behavior It can be a fish bee dinosaur etc

bull Rules for flockingndash Cohesion Each boid fly towards the centroid of its

local flock mates (that is boid in its local neighborhood)ndash Separation Each boid keep a certain distance away

from local flock mates to avoid collisionsndash Alignment Each boid align its velocity vector and keep

velocity magnitude similar with that of the local flock

23418 40

bull Imagine a birdrsquos flock in an area where there is a single food source

bull A bird donrsquot know where the food is but it knows its distance to the food

bull The best strategy is to follow the bird that is closer to the food

bull Particles save and communicate the best solution they have found

From Bird to Particle

23418 41

Features of PSO

bull Population initialized by assigning random positions and velocities potential solutions are then flown through hyperspace

bull Each particle keeps track of its ldquobestrdquo (highest fitness) position in hyperspacendash This is called ldquopBestrdquo for an individual particlendash It is called ldquogBestrdquo for the best in the populationndash It is called ldquolBestrdquo for the best in a defined neighborhood

bull At each time step each particle stochastically accelerates toward its pBest and gBest (or lBest)

23418 42

Particle Swarm Optimization Process

Step1 Initialize population in hyperspace

Step2 Evaluate fitness of individual particles

Step3 Modify velocities based on previous best and global (or neighborhood) best

Step4 Terminate on some condition

Step5 Go to step 2

23418 43

How do particles fly

bull Combination of gBest and the pBest (lBest)ndash need a compromise

bull lBest can bendash Social the particles around are always the same no matter

where they are in spacendash Geographical the particles around are those whose distance is

the shortest

bull Global PSO vs Local PSOndash the global version converges quickly to a solution but it gets

more easily stuck in local minima

23418 44

Illustrating the velocity update schema of global PSO

23418 45

PSO Related issues

bull Controlling velocities (determining the best value for Vmax)ndash Usually set maximum velocity to dynamic range of variable

bull Usually set c1 and c2 to 2

bull Inertia weight influences tradeoff between global and local explorationndash Good approach is to reduce inertia weight during run (ie from

09 to 04 over 1000 generations)

bull Swarm Size and Neighborhood Size

23418 46

Advantages of PSO

bull Adaptation operates on velocitiesndash Most other methods operate on positionsndash Effect PSO has a builtin momentumndash Particles tend to hurdle past optima ndash an advantage

since the best positions are remembered anyway

bull Simple in conceptbull Easy to implementbull Computationally efficientbull Effective on a variety of problems

23418 47

Summarybull Swarm Intelligence (SI)

ndash an artificial intelligence technique based around the study of collective behavior in decentralized self-organized systems

bull Multi-Agent Systems (MAS)ndash A system that consists of a number of agents

which interact with one-anotherndash Communication Coordination

Collaboration

23418 48

bull Ant Colony Optimization (ACO)ndash Inspired by ant colony foragingndash pheromone as heuristic information

(stigmergy)ndash Iteration between ConstructAntSolutions and

UpdatePheromones

bull Particle Swarm Optimization (PSO)ndash Inspired by bird flockingndash Heuristic information results from partnersndash Particle Velocity Update

• Collective Intelligence
• Outline
• The Computational Beauty of Nature
• Examples of Collective Behavior in Nature and Society
• Emergence
• Biological motivation Insect Societies
• Insect Societies
• Self Organization
• Stigmergy
• Flocks Herds and Schools
• Boid rules
• Slide 12
• Swarm Intelligence (Contrsquod)
• Components of SI
• Characteristics of SI
• The Web becomes a Giant Brain Some see the Web evolving into a collective brain for humankind
• What is Multi-agent Systems
• What is MAS(Contrsquod)
• Slide 19
• Communication models
• Working together
• Example mechanism Contract Net Protocol (CNP)
• Slide 23
• CNP (Contrsquod)
• Slide 25
• Attributes of Multi-agent Systems
• Applications of MAS
• Applications of MAS (Contrsquod)
• How to simulate SI for search
• Part Ⅱ Ant Colony Optimization (ACO)
• Natural Ants
• Slide 32
• How can the natural ants find the shortest path
• Slide 34
• Problems of AS
• Part Ⅲ Particle Swarm Optimization (PSO)
• Slide 37
• Modeling bird flocking
• Modeling bird flocking (Contrsquod)
• From Bird to Particle
• Features of PSO
• Particle Swarm Optimization Process
• How do particles fly
• Slide 44
• PSO Related issues
• Advantages of PSO
• Summary
• Slide 48

23418 7

bull Collective systems capable of accomplishing difficult tasks in dynamic and varied environments without any external guidance or control and with no central coordination

bull Achieving a collective performance which could not normally be achieved by any individual acting alone

bull The colony as a whole is the seat of a stable and self-regulated organization of individual behavior which adapts itself very easily to the unpredictable characteristics of the environment within which it evolved

Insect Societies

23418 8

Self Organizationbull Insect societies have developed systems of collective

decision making operating without symbolic representations exploiting the physical constraints of the environment in which they evolved and using communications between individuals either directly when in contact or indirectly (stigmergy) using the environment as a channel of communication

bull Through these direct and indirect interactions the society self organizes and faced with a problem finds a solution with a complexity far greater than that of the insects of which it is composed

23418 9

Stigmergy

bull Indirect communication via interaction with environment [Gasseacute 59]ndash Sematonic [Wilson 75] stigmergy

bull action of agent directly related to problem solving and affects behavior of other agents

ndash Sign-based stigmergybull action of agent affects environment not directly

related to problem solving activity

23418 10

Flocks Herds and Schoolsbull In the late 80rsquos Craig Reynolds created a simple model of animal motion that he called Boids

bull Itrsquos generates very realistic motion for movement from three simple rules which define a boidrsquos steering behaviour

bull This model and its variations has been used to drive animations of birds insects people fish antelope etc in films (eg Batman Returns Lion King)

23418 11

Boid rulesSeparation steer to avoid crowding local flockmates

- A fundamental rule that has priority over the others- Also useful in avoiding collisions with other objects in the environment

Alignment steer towards the average heading and speed of local flockmates

- Enforces cohesion to keep the flock togetherHelps with collision avoidance too

Cohesion steer to move toward the average position of local flockmates

- Agents at edge of the herd more vulnerable to predators

- Helps to keep the flock together

23418 12

bull Swarm intelligence (SI) is an artificial intelligence technique based around the study of collective behavior in decentralized self-organized systems

bull The express lsquoSwarm Intelligencersquo was originally used by Beni Hackwood and Wang in 1989 in the context of cellular robotic systems to describe the self-organization of simple mechanical agents

bull It was later extended to include ldquoany attempt to design algorithms or distributed problem-solving devices inspired by the collective behavior of social insect colonies and other animal societiesrdquo [Bonabeau Dorigo and Theraulaz 1999]

Swarm Intelligence

23418 13

Swarm Intelligence (Contrsquod)

bull SI systems are typically made up of a population of simple agents interacting locally with one another and with their environment

bull Although there is normally no centralized control structure dictating how individual agents should behave local interactions between such agents often lead to the emergence of global behavior

bull Sometimes called lsquoCollective Intelligencersquo

23418 14

Components of SIbull Agents

ndash Interact with the world and with each other (either directly or indirectly)

bull Simple behavioursndash eg ants termites bees wasps

bull Communicationndash How agents interact with each otherndash eg pheromones of ants

Simple behaviours of individual agents

+ Communication between a group of agents

= Emergent complex behaviour of the group of agents

23418 15

Characteristics of SI

bull Distributed no central control or data source

bull Limited communication

bull No (explicit) model of the environment

bull Perception of environment (sensing)

bull Ability to react to environment changes

Is SI relevant to people

23418 16

The Web becomes a Giant Brain

Some see the Web evolving

intoa collective

brain for humankind

23418 17

What is Multi-agent Systemsbull A set of agents which interact in a common environmentbull Focus on the collaborative resolution of global

problems by a set of distributed entitiesbull Agent attempt to satisfy their own local goals as well as

the collaborative global goals bull To successfully interact they will require the ability to

cooperate coordinate and negotiate with each other much as people do

23418 18

What is MAS(Contrsquod)

bull MAS as seen from Distributed AIndash a loosely coupled network of entities that work

together to find answers to problems that are beyond the individual capabilities or knowledge of each entity

bull A more general meaningndash systems composed of autonomous components

that exhibit the following characteristicsbull each agent has incomplete capabilities to solve a problembull there is no global system controlbull data is decentralizedbull computation is asynchronous

23418 19

bull Traditionalndash Client-serverndash Low-level

messagesndash Synchronousndash Can not do the job

bull Agent breakthroughsndash Peer-to-peer

topologyndash Blackboard

coordination modelndash Encapsulated

messagingndash High-level message

protocols

Client ServerFunction(Parameters)

Return(Parameters)

Traditional Software

IntelligentAgents

IntelligentAgents

IntelligentAgents

Blackboard

MessageReply

Agents IntelligentAgentsIntelligent

Agents

IntelligentAgentsIntelligent

Agents

IntelligentAgentsIntelligent

Agents

23418 20

Communication models

bull Theoretical models Speech act theory

bull Practical modelsndash shared languages like KIF KQML DAMLndash service models like DAML-Sndash social convention protocols

23418 21

Working together

bull Benevolent Agentsndash assume agents are benevolent our best

interest is their best interest

bull Self-Interested Agentsndash Agents will be assumed to act to further their

own interests possibly at expense of othersndash Potential for conflict

23418 22

Example mechanism Contract Net Protocol (CNP)

bull Negotiation as a collaboration mechanism

bull Negotiation on how tasks should be shared

ndash A task (plan) may be decomposed in a hierarchy of subtasks (hierarchical planning)

ndash An agent may subcontract another agent to perform a (sub)task

Contract

Bid

agent agent

CNP

Task announceme

nt

23418 24

CNP (Contrsquod)

Contractor

Potential candidate agents

Task announcement (broadcast)

Contractor

Candidate Candidate

Bid

Bid

Phase 1 Task Announcement

- The contractor agent publicly announces a task

- Potential candidates evaluate the task according to their won skills and availability

Phase 2 Submission of Bids Proposals

- Agents that satisfy the requiremenst ie are able to perform the task send their bid proposal to the contractor

23418 25

CNP (Contrsquod)

Contractor

Selected candidate

Contractor

Contracted agent

Contract

Phase 3 Selection

- The selection of the best candidate is made by the contractor based on received bids and on the CVs of the candidates

Phase 4 Contract awarding

- A contract is established between the contractor and the selected candidate

- A privileged bilateral communication channel is established between the two agents

23418 26

Attributes of Multi-agent Systems

Apply MAS when some of the following features show up in a problem

bull Decentralizationbull Complex components often best described at

the knowledge levelbull Adaptive behaviorbull Complex interactionsbull Coordinationbull Emergent aggregate behaviors

23418 27

Applications of MAS

Advanced Manufacturing Management Systemsminus Agents as representatives of machines users business

processes etc

Intelligent Information Search on Internetminus Some agents may show learning capabilities (learn the

preferences of their users )

Intelligent security enforcement on Internetminus Agents are representative of sensors or IDSs

Shopping Agents in Electronic Commerceminus With search price comparison and bargaining capabilities

23418 28

Applications of MAS (Contrsquod)

Multi-agent auction in E-commerce

Distributed Surveillanceminus For information search or to look for special events informing

their users of relevant news

Distributed Signal Processingminus For problem diagnosis situation assessment etc in the

network

Distributed Problem Solvingminus Collaborative design scheduling and planning

23418 29

How to simulate SI for search

Example1 Ant --gt Ant Colony Optimization (ACO)

Example2 Bird Flocking --gt Particle Swarm Optimization (PSO)

23418 30

Part Ant Colony Optimization Ⅱ(ACO)

First proposed by M Dorigo 1992

23418 31

Natural Antsbull Individual ants are simple insects with

limited memory and capable of performing simple actions

bull However an ant colony expresses a complex collective behavior providing intelligent solutions to problems such asndash carrying large itemsndash forming bridgesndash finding the shortest routes from the nest

to a food source prioritizing food sources based on their distance and ease of access

23418 32

Natural Antsbull Moreover in a colony each ant has its prescribed task but

the ants can switch tasks if the collective needs it ndash Outside the nest ants can have 4 different tasks

bull Foraging searching for and retrieving food

bull Patrolling looking for food supply

bull Midden work Sorting the colony refuse pile

bull Nest maintenance work construction and clearing of chambers

ndash An antrsquos decision whether to perform a task depends onbull The Phisical State of the environment

ndash If part of the nest is damaged more ants do nest maintenance work to repair it

bull Social Interactions with other ants

Communication (direct or indirect) is necessary

23418 33

bull They establish indirect communication system based on the deposition of pheromone over the path they followndash An isolated ant moves at random but when it finds a pheromone

trail there is a high probability that this ant will decide to follow the trail

ndash An ant foraging for food deposits pheromone over its route When it finds a food source it returns to the nest reinforcing its trail

ndash So other ants have greater probability to start following this trail and thereby laying more pheromone on it

ndash This process works as a positive feedback loop system because the higher the intensity of the pheromone over a trail the higher the probability of an ant start traveling through it

How can the natural ants find the shortest path

23418 34

bull Since the route B is shorter the ants on this path will complete the travel more times and thereby lay more pheromone over it

bull The pheromone concentration on trail B will increase at a higher rate than on A and soon the ants on route A will choose to follow route B

bull Since most ants will no longer travel on route A and since the pheromone is volatile trail A will start evaporating

bull Only the shortest route will remain

23418 35

Problems of ASbull Ant System tends to converge quickly

ndash This means that its exploitation of the best solution found is too high it should be exploring solution space more

ndash Pheromone evaporationupdate rule (better rule may exist)

bull Led to extensions of the ant systemndash Elitist Strategy for Ant Systems (EAS)ndash Rank based Ant Systems (AKRANK)ndash MAX-MIN Ant system (MMAS)ndash Ant Colony System (ACS)

23418 36

Part Ⅲ Particle Swarm Optimization (PSO)

1048713Firstly Proposed by Kennedy and Eberhart 1995

ldquoOnce again nature has provided us with a technique for processing information that is at once elegant and versatilerdquo

23418 37

bull In PSO a ldquoswarmrdquo is defined as an apparently disorganized collection (population) of moving individuals that tend to cluster together while each individual seems to be moving in a random direction

Bird flocking is one of the best example of PSO in nature

23418 38

Modeling bird flocking

bull The synchrony of flocking behavior is thought to be a function of birdrsquos efforts to maintain an optimal distance between themselves and their neighborsndash Birds and fish adjust their physical movement to avoid

predators seek for food and mates ndash Humans tend to adjust our beliefs and attitudes to

conform with those of our social peers Humans change in abstract multidimensional space collision-free

23418 39

Modeling bird flocking (Contrsquod)

bull Definitions ndash Flock is a group of objects that exhibit the general

class of polarized (aligned) non-colliding aggregate motion

ndash Boid is a simulated bird-like object ie it exhibits this type of behavior It can be a fish bee dinosaur etc

bull Rules for flockingndash Cohesion Each boid fly towards the centroid of its

local flock mates (that is boid in its local neighborhood)ndash Separation Each boid keep a certain distance away

from local flock mates to avoid collisionsndash Alignment Each boid align its velocity vector and keep

velocity magnitude similar with that of the local flock

23418 40

bull Imagine a birdrsquos flock in an area where there is a single food source

bull A bird donrsquot know where the food is but it knows its distance to the food

bull The best strategy is to follow the bird that is closer to the food

bull Particles save and communicate the best solution they have found

From Bird to Particle

23418 41

Features of PSO

bull Population initialized by assigning random positions and velocities potential solutions are then flown through hyperspace

bull Each particle keeps track of its ldquobestrdquo (highest fitness) position in hyperspacendash This is called ldquopBestrdquo for an individual particlendash It is called ldquogBestrdquo for the best in the populationndash It is called ldquolBestrdquo for the best in a defined neighborhood

bull At each time step each particle stochastically accelerates toward its pBest and gBest (or lBest)

23418 42

Particle Swarm Optimization Process

Step1 Initialize population in hyperspace

Step2 Evaluate fitness of individual particles

Step3 Modify velocities based on previous best and global (or neighborhood) best

Step4 Terminate on some condition

Step5 Go to step 2

23418 43

How do particles fly

bull Combination of gBest and the pBest (lBest)ndash need a compromise

bull lBest can bendash Social the particles around are always the same no matter

where they are in spacendash Geographical the particles around are those whose distance is

the shortest

bull Global PSO vs Local PSOndash the global version converges quickly to a solution but it gets

more easily stuck in local minima

23418 44

Illustrating the velocity update schema of global PSO

23418 45

PSO Related issues

bull Controlling velocities (determining the best value for Vmax)ndash Usually set maximum velocity to dynamic range of variable

bull Usually set c1 and c2 to 2

bull Inertia weight influences tradeoff between global and local explorationndash Good approach is to reduce inertia weight during run (ie from

09 to 04 over 1000 generations)

bull Swarm Size and Neighborhood Size

23418 46

Advantages of PSO

bull Adaptation operates on velocitiesndash Most other methods operate on positionsndash Effect PSO has a builtin momentumndash Particles tend to hurdle past optima ndash an advantage

since the best positions are remembered anyway

bull Simple in conceptbull Easy to implementbull Computationally efficientbull Effective on a variety of problems

23418 47

Summarybull Swarm Intelligence (SI)

ndash an artificial intelligence technique based around the study of collective behavior in decentralized self-organized systems

bull Multi-Agent Systems (MAS)ndash A system that consists of a number of agents

which interact with one-anotherndash Communication Coordination

Collaboration

23418 48

bull Ant Colony Optimization (ACO)ndash Inspired by ant colony foragingndash pheromone as heuristic information

(stigmergy)ndash Iteration between ConstructAntSolutions and

UpdatePheromones

bull Particle Swarm Optimization (PSO)ndash Inspired by bird flockingndash Heuristic information results from partnersndash Particle Velocity Update

• Collective Intelligence
• Outline
• The Computational Beauty of Nature
• Examples of Collective Behavior in Nature and Society
• Emergence
• Biological motivation Insect Societies
• Insect Societies
• Self Organization
• Stigmergy
• Flocks Herds and Schools
• Boid rules
• Slide 12
• Swarm Intelligence (Contrsquod)
• Components of SI
• Characteristics of SI
• The Web becomes a Giant Brain Some see the Web evolving into a collective brain for humankind
• What is Multi-agent Systems
• What is MAS(Contrsquod)
• Slide 19
• Communication models
• Working together
• Example mechanism Contract Net Protocol (CNP)
• Slide 23
• CNP (Contrsquod)
• Slide 25
• Attributes of Multi-agent Systems
• Applications of MAS
• Applications of MAS (Contrsquod)
• How to simulate SI for search
• Part Ⅱ Ant Colony Optimization (ACO)
• Natural Ants
• Slide 32
• How can the natural ants find the shortest path
• Slide 34
• Problems of AS
• Part Ⅲ Particle Swarm Optimization (PSO)
• Slide 37
• Modeling bird flocking
• Modeling bird flocking (Contrsquod)
• From Bird to Particle
• Features of PSO
• Particle Swarm Optimization Process
• How do particles fly
• Slide 44
• PSO Related issues
• Advantages of PSO
• Summary
• Slide 48

23418 8

Self Organizationbull Insect societies have developed systems of collective

decision making operating without symbolic representations exploiting the physical constraints of the environment in which they evolved and using communications between individuals either directly when in contact or indirectly (stigmergy) using the environment as a channel of communication

bull Through these direct and indirect interactions the society self organizes and faced with a problem finds a solution with a complexity far greater than that of the insects of which it is composed

23418 9

Stigmergy

bull Indirect communication via interaction with environment [Gasseacute 59]ndash Sematonic [Wilson 75] stigmergy

bull action of agent directly related to problem solving and affects behavior of other agents

ndash Sign-based stigmergybull action of agent affects environment not directly

related to problem solving activity

23418 10

Flocks Herds and Schoolsbull In the late 80rsquos Craig Reynolds created a simple model of animal motion that he called Boids

bull Itrsquos generates very realistic motion for movement from three simple rules which define a boidrsquos steering behaviour

bull This model and its variations has been used to drive animations of birds insects people fish antelope etc in films (eg Batman Returns Lion King)

23418 11

Boid rulesSeparation steer to avoid crowding local flockmates

- A fundamental rule that has priority over the others- Also useful in avoiding collisions with other objects in the environment

Alignment steer towards the average heading and speed of local flockmates

- Enforces cohesion to keep the flock togetherHelps with collision avoidance too

Cohesion steer to move toward the average position of local flockmates

- Agents at edge of the herd more vulnerable to predators

- Helps to keep the flock together

23418 12

bull Swarm intelligence (SI) is an artificial intelligence technique based around the study of collective behavior in decentralized self-organized systems

bull The express lsquoSwarm Intelligencersquo was originally used by Beni Hackwood and Wang in 1989 in the context of cellular robotic systems to describe the self-organization of simple mechanical agents

bull It was later extended to include ldquoany attempt to design algorithms or distributed problem-solving devices inspired by the collective behavior of social insect colonies and other animal societiesrdquo [Bonabeau Dorigo and Theraulaz 1999]

Swarm Intelligence

23418 13

Swarm Intelligence (Contrsquod)

bull SI systems are typically made up of a population of simple agents interacting locally with one another and with their environment

bull Although there is normally no centralized control structure dictating how individual agents should behave local interactions between such agents often lead to the emergence of global behavior

bull Sometimes called lsquoCollective Intelligencersquo

23418 14

Components of SIbull Agents

ndash Interact with the world and with each other (either directly or indirectly)

bull Simple behavioursndash eg ants termites bees wasps

bull Communicationndash How agents interact with each otherndash eg pheromones of ants

Simple behaviours of individual agents

+ Communication between a group of agents

= Emergent complex behaviour of the group of agents

23418 15

Characteristics of SI

bull Distributed no central control or data source

bull Limited communication

bull No (explicit) model of the environment

bull Perception of environment (sensing)

bull Ability to react to environment changes

Is SI relevant to people

23418 16

The Web becomes a Giant Brain

Some see the Web evolving

intoa collective

brain for humankind

23418 17

What is Multi-agent Systemsbull A set of agents which interact in a common environmentbull Focus on the collaborative resolution of global

problems by a set of distributed entitiesbull Agent attempt to satisfy their own local goals as well as

the collaborative global goals bull To successfully interact they will require the ability to

cooperate coordinate and negotiate with each other much as people do

23418 18

What is MAS(Contrsquod)

bull MAS as seen from Distributed AIndash a loosely coupled network of entities that work

together to find answers to problems that are beyond the individual capabilities or knowledge of each entity

bull A more general meaningndash systems composed of autonomous components

that exhibit the following characteristicsbull each agent has incomplete capabilities to solve a problembull there is no global system controlbull data is decentralizedbull computation is asynchronous

23418 19

bull Traditionalndash Client-serverndash Low-level

messagesndash Synchronousndash Can not do the job

bull Agent breakthroughsndash Peer-to-peer

topologyndash Blackboard

coordination modelndash Encapsulated

messagingndash High-level message

protocols

Client ServerFunction(Parameters)

Return(Parameters)

Traditional Software

IntelligentAgents

IntelligentAgents

IntelligentAgents

Blackboard

MessageReply

Agents IntelligentAgentsIntelligent

Agents

IntelligentAgentsIntelligent

Agents

IntelligentAgentsIntelligent

Agents

23418 20

Communication models

bull Theoretical models Speech act theory

bull Practical modelsndash shared languages like KIF KQML DAMLndash service models like DAML-Sndash social convention protocols

23418 21

Working together

bull Benevolent Agentsndash assume agents are benevolent our best

interest is their best interest

bull Self-Interested Agentsndash Agents will be assumed to act to further their

own interests possibly at expense of othersndash Potential for conflict

23418 22

Example mechanism Contract Net Protocol (CNP)

bull Negotiation as a collaboration mechanism

bull Negotiation on how tasks should be shared

ndash A task (plan) may be decomposed in a hierarchy of subtasks (hierarchical planning)

ndash An agent may subcontract another agent to perform a (sub)task

Contract

Bid

agent agent

CNP

Task announceme

nt

23418 24

CNP (Contrsquod)

Contractor

Potential candidate agents

Task announcement (broadcast)

Contractor

Candidate Candidate

Bid

Bid

Phase 1 Task Announcement

- The contractor agent publicly announces a task

- Potential candidates evaluate the task according to their won skills and availability

Phase 2 Submission of Bids Proposals

- Agents that satisfy the requiremenst ie are able to perform the task send their bid proposal to the contractor

23418 25

CNP (Contrsquod)

Contractor

Selected candidate

Contractor

Contracted agent

Contract

Phase 3 Selection

- The selection of the best candidate is made by the contractor based on received bids and on the CVs of the candidates

Phase 4 Contract awarding

- A contract is established between the contractor and the selected candidate

- A privileged bilateral communication channel is established between the two agents

23418 26

Attributes of Multi-agent Systems

Apply MAS when some of the following features show up in a problem

bull Decentralizationbull Complex components often best described at

the knowledge levelbull Adaptive behaviorbull Complex interactionsbull Coordinationbull Emergent aggregate behaviors

23418 27

Applications of MAS

Advanced Manufacturing Management Systemsminus Agents as representatives of machines users business

processes etc

Intelligent Information Search on Internetminus Some agents may show learning capabilities (learn the

preferences of their users )

Intelligent security enforcement on Internetminus Agents are representative of sensors or IDSs

Shopping Agents in Electronic Commerceminus With search price comparison and bargaining capabilities

23418 28

Applications of MAS (Contrsquod)

Multi-agent auction in E-commerce

Distributed Surveillanceminus For information search or to look for special events informing

their users of relevant news

Distributed Signal Processingminus For problem diagnosis situation assessment etc in the

network

Distributed Problem Solvingminus Collaborative design scheduling and planning

23418 29

How to simulate SI for search

Example1 Ant --gt Ant Colony Optimization (ACO)

Example2 Bird Flocking --gt Particle Swarm Optimization (PSO)

23418 30

Part Ant Colony Optimization Ⅱ(ACO)

First proposed by M Dorigo 1992

23418 31

Natural Antsbull Individual ants are simple insects with

limited memory and capable of performing simple actions

bull However an ant colony expresses a complex collective behavior providing intelligent solutions to problems such asndash carrying large itemsndash forming bridgesndash finding the shortest routes from the nest

to a food source prioritizing food sources based on their distance and ease of access

23418 32

Natural Antsbull Moreover in a colony each ant has its prescribed task but

the ants can switch tasks if the collective needs it ndash Outside the nest ants can have 4 different tasks

bull Foraging searching for and retrieving food

bull Patrolling looking for food supply

bull Midden work Sorting the colony refuse pile

bull Nest maintenance work construction and clearing of chambers

ndash An antrsquos decision whether to perform a task depends onbull The Phisical State of the environment

ndash If part of the nest is damaged more ants do nest maintenance work to repair it

bull Social Interactions with other ants

Communication (direct or indirect) is necessary

23418 33

bull They establish indirect communication system based on the deposition of pheromone over the path they followndash An isolated ant moves at random but when it finds a pheromone

trail there is a high probability that this ant will decide to follow the trail

ndash An ant foraging for food deposits pheromone over its route When it finds a food source it returns to the nest reinforcing its trail

ndash So other ants have greater probability to start following this trail and thereby laying more pheromone on it

ndash This process works as a positive feedback loop system because the higher the intensity of the pheromone over a trail the higher the probability of an ant start traveling through it

How can the natural ants find the shortest path

23418 34

bull Since the route B is shorter the ants on this path will complete the travel more times and thereby lay more pheromone over it

bull The pheromone concentration on trail B will increase at a higher rate than on A and soon the ants on route A will choose to follow route B

bull Since most ants will no longer travel on route A and since the pheromone is volatile trail A will start evaporating

bull Only the shortest route will remain

23418 35

Problems of ASbull Ant System tends to converge quickly

ndash This means that its exploitation of the best solution found is too high it should be exploring solution space more

ndash Pheromone evaporationupdate rule (better rule may exist)

bull Led to extensions of the ant systemndash Elitist Strategy for Ant Systems (EAS)ndash Rank based Ant Systems (AKRANK)ndash MAX-MIN Ant system (MMAS)ndash Ant Colony System (ACS)

23418 36

Part Ⅲ Particle Swarm Optimization (PSO)

1048713Firstly Proposed by Kennedy and Eberhart 1995

ldquoOnce again nature has provided us with a technique for processing information that is at once elegant and versatilerdquo

23418 37

bull In PSO a ldquoswarmrdquo is defined as an apparently disorganized collection (population) of moving individuals that tend to cluster together while each individual seems to be moving in a random direction

Bird flocking is one of the best example of PSO in nature

23418 38

Modeling bird flocking

bull The synchrony of flocking behavior is thought to be a function of birdrsquos efforts to maintain an optimal distance between themselves and their neighborsndash Birds and fish adjust their physical movement to avoid

predators seek for food and mates ndash Humans tend to adjust our beliefs and attitudes to

conform with those of our social peers Humans change in abstract multidimensional space collision-free

23418 39

Modeling bird flocking (Contrsquod)

bull Definitions ndash Flock is a group of objects that exhibit the general

class of polarized (aligned) non-colliding aggregate motion

ndash Boid is a simulated bird-like object ie it exhibits this type of behavior It can be a fish bee dinosaur etc

bull Rules for flockingndash Cohesion Each boid fly towards the centroid of its

local flock mates (that is boid in its local neighborhood)ndash Separation Each boid keep a certain distance away

from local flock mates to avoid collisionsndash Alignment Each boid align its velocity vector and keep

velocity magnitude similar with that of the local flock

23418 40

bull Imagine a birdrsquos flock in an area where there is a single food source

bull A bird donrsquot know where the food is but it knows its distance to the food

bull The best strategy is to follow the bird that is closer to the food

bull Particles save and communicate the best solution they have found

From Bird to Particle

23418 41

Features of PSO

bull Population initialized by assigning random positions and velocities potential solutions are then flown through hyperspace

bull Each particle keeps track of its ldquobestrdquo (highest fitness) position in hyperspacendash This is called ldquopBestrdquo for an individual particlendash It is called ldquogBestrdquo for the best in the populationndash It is called ldquolBestrdquo for the best in a defined neighborhood

bull At each time step each particle stochastically accelerates toward its pBest and gBest (or lBest)

23418 42

Particle Swarm Optimization Process

Step1 Initialize population in hyperspace

Step2 Evaluate fitness of individual particles

Step3 Modify velocities based on previous best and global (or neighborhood) best

Step4 Terminate on some condition

Step5 Go to step 2

23418 43

How do particles fly

bull Combination of gBest and the pBest (lBest)ndash need a compromise

bull lBest can bendash Social the particles around are always the same no matter

where they are in spacendash Geographical the particles around are those whose distance is

the shortest

bull Global PSO vs Local PSOndash the global version converges quickly to a solution but it gets

more easily stuck in local minima

23418 44

Illustrating the velocity update schema of global PSO

23418 45

PSO Related issues

bull Controlling velocities (determining the best value for Vmax)ndash Usually set maximum velocity to dynamic range of variable

bull Usually set c1 and c2 to 2

bull Inertia weight influences tradeoff between global and local explorationndash Good approach is to reduce inertia weight during run (ie from

09 to 04 over 1000 generations)

bull Swarm Size and Neighborhood Size

23418 46

Advantages of PSO

bull Adaptation operates on velocitiesndash Most other methods operate on positionsndash Effect PSO has a builtin momentumndash Particles tend to hurdle past optima ndash an advantage

since the best positions are remembered anyway

bull Simple in conceptbull Easy to implementbull Computationally efficientbull Effective on a variety of problems

23418 47

Summarybull Swarm Intelligence (SI)

ndash an artificial intelligence technique based around the study of collective behavior in decentralized self-organized systems

bull Multi-Agent Systems (MAS)ndash A system that consists of a number of agents

which interact with one-anotherndash Communication Coordination

Collaboration

23418 48

bull Ant Colony Optimization (ACO)ndash Inspired by ant colony foragingndash pheromone as heuristic information

(stigmergy)ndash Iteration between ConstructAntSolutions and

UpdatePheromones

bull Particle Swarm Optimization (PSO)ndash Inspired by bird flockingndash Heuristic information results from partnersndash Particle Velocity Update

• Collective Intelligence
• Outline
• The Computational Beauty of Nature
• Examples of Collective Behavior in Nature and Society
• Emergence
• Biological motivation Insect Societies
• Insect Societies
• Self Organization
• Stigmergy
• Flocks Herds and Schools
• Boid rules
• Slide 12
• Swarm Intelligence (Contrsquod)
• Components of SI
• Characteristics of SI
• The Web becomes a Giant Brain Some see the Web evolving into a collective brain for humankind
• What is Multi-agent Systems
• What is MAS(Contrsquod)
• Slide 19
• Communication models
• Working together
• Example mechanism Contract Net Protocol (CNP)
• Slide 23
• CNP (Contrsquod)
• Slide 25
• Attributes of Multi-agent Systems
• Applications of MAS
• Applications of MAS (Contrsquod)
• How to simulate SI for search
• Part Ⅱ Ant Colony Optimization (ACO)
• Natural Ants
• Slide 32
• How can the natural ants find the shortest path
• Slide 34
• Problems of AS
• Part Ⅲ Particle Swarm Optimization (PSO)
• Slide 37
• Modeling bird flocking
• Modeling bird flocking (Contrsquod)
• From Bird to Particle
• Features of PSO
• Particle Swarm Optimization Process
• How do particles fly
• Slide 44
• PSO Related issues
• Advantages of PSO
• Summary
• Slide 48

23418 9

Stigmergy

bull Indirect communication via interaction with environment [Gasseacute 59]ndash Sematonic [Wilson 75] stigmergy

bull action of agent directly related to problem solving and affects behavior of other agents

ndash Sign-based stigmergybull action of agent affects environment not directly

related to problem solving activity

23418 10

Flocks Herds and Schoolsbull In the late 80rsquos Craig Reynolds created a simple model of animal motion that he called Boids

bull Itrsquos generates very realistic motion for movement from three simple rules which define a boidrsquos steering behaviour

bull This model and its variations has been used to drive animations of birds insects people fish antelope etc in films (eg Batman Returns Lion King)

23418 11

Boid rulesSeparation steer to avoid crowding local flockmates

- A fundamental rule that has priority over the others- Also useful in avoiding collisions with other objects in the environment

Alignment steer towards the average heading and speed of local flockmates

- Enforces cohesion to keep the flock togetherHelps with collision avoidance too

Cohesion steer to move toward the average position of local flockmates

- Agents at edge of the herd more vulnerable to predators

- Helps to keep the flock together

23418 12

bull Swarm intelligence (SI) is an artificial intelligence technique based around the study of collective behavior in decentralized self-organized systems

bull The express lsquoSwarm Intelligencersquo was originally used by Beni Hackwood and Wang in 1989 in the context of cellular robotic systems to describe the self-organization of simple mechanical agents

bull It was later extended to include ldquoany attempt to design algorithms or distributed problem-solving devices inspired by the collective behavior of social insect colonies and other animal societiesrdquo [Bonabeau Dorigo and Theraulaz 1999]

Swarm Intelligence

23418 13

Swarm Intelligence (Contrsquod)

bull SI systems are typically made up of a population of simple agents interacting locally with one another and with their environment

bull Although there is normally no centralized control structure dictating how individual agents should behave local interactions between such agents often lead to the emergence of global behavior

bull Sometimes called lsquoCollective Intelligencersquo

23418 14

Components of SIbull Agents

ndash Interact with the world and with each other (either directly or indirectly)

bull Simple behavioursndash eg ants termites bees wasps

bull Communicationndash How agents interact with each otherndash eg pheromones of ants

Simple behaviours of individual agents

+ Communication between a group of agents

= Emergent complex behaviour of the group of agents

23418 15

Characteristics of SI

bull Distributed no central control or data source

bull Limited communication

bull No (explicit) model of the environment

bull Perception of environment (sensing)

bull Ability to react to environment changes

Is SI relevant to people

23418 16

The Web becomes a Giant Brain

Some see the Web evolving

intoa collective

brain for humankind

23418 17

What is Multi-agent Systemsbull A set of agents which interact in a common environmentbull Focus on the collaborative resolution of global

problems by a set of distributed entitiesbull Agent attempt to satisfy their own local goals as well as

the collaborative global goals bull To successfully interact they will require the ability to

cooperate coordinate and negotiate with each other much as people do

23418 18

What is MAS(Contrsquod)

bull MAS as seen from Distributed AIndash a loosely coupled network of entities that work

together to find answers to problems that are beyond the individual capabilities or knowledge of each entity

bull A more general meaningndash systems composed of autonomous components

that exhibit the following characteristicsbull each agent has incomplete capabilities to solve a problembull there is no global system controlbull data is decentralizedbull computation is asynchronous

23418 19

bull Traditionalndash Client-serverndash Low-level

messagesndash Synchronousndash Can not do the job

bull Agent breakthroughsndash Peer-to-peer

topologyndash Blackboard

coordination modelndash Encapsulated

messagingndash High-level message

protocols

Client ServerFunction(Parameters)

Return(Parameters)

Traditional Software

IntelligentAgents

IntelligentAgents

IntelligentAgents

Blackboard

MessageReply

Agents IntelligentAgentsIntelligent

Agents

IntelligentAgentsIntelligent

Agents

IntelligentAgentsIntelligent

Agents

23418 20

Communication models

bull Theoretical models Speech act theory

bull Practical modelsndash shared languages like KIF KQML DAMLndash service models like DAML-Sndash social convention protocols

23418 21

Working together

bull Benevolent Agentsndash assume agents are benevolent our best

interest is their best interest

bull Self-Interested Agentsndash Agents will be assumed to act to further their

own interests possibly at expense of othersndash Potential for conflict

23418 22

Example mechanism Contract Net Protocol (CNP)

bull Negotiation as a collaboration mechanism

bull Negotiation on how tasks should be shared

ndash A task (plan) may be decomposed in a hierarchy of subtasks (hierarchical planning)

ndash An agent may subcontract another agent to perform a (sub)task

Contract

Bid

agent agent

CNP

Task announceme

nt

23418 24

CNP (Contrsquod)

Contractor

Potential candidate agents

Task announcement (broadcast)

Contractor

Candidate Candidate

Bid

Bid

Phase 1 Task Announcement

- The contractor agent publicly announces a task

- Potential candidates evaluate the task according to their won skills and availability

Phase 2 Submission of Bids Proposals

- Agents that satisfy the requiremenst ie are able to perform the task send their bid proposal to the contractor

23418 25

CNP (Contrsquod)

Contractor

Selected candidate

Contractor

Contracted agent

Contract

Phase 3 Selection

- The selection of the best candidate is made by the contractor based on received bids and on the CVs of the candidates

Phase 4 Contract awarding

- A contract is established between the contractor and the selected candidate

- A privileged bilateral communication channel is established between the two agents

23418 26

Attributes of Multi-agent Systems

Apply MAS when some of the following features show up in a problem

bull Decentralizationbull Complex components often best described at

the knowledge levelbull Adaptive behaviorbull Complex interactionsbull Coordinationbull Emergent aggregate behaviors

23418 27

Applications of MAS

Advanced Manufacturing Management Systemsminus Agents as representatives of machines users business

processes etc

Intelligent Information Search on Internetminus Some agents may show learning capabilities (learn the

preferences of their users )

Intelligent security enforcement on Internetminus Agents are representative of sensors or IDSs

Shopping Agents in Electronic Commerceminus With search price comparison and bargaining capabilities

23418 28

Applications of MAS (Contrsquod)

Multi-agent auction in E-commerce

Distributed Surveillanceminus For information search or to look for special events informing

their users of relevant news

Distributed Signal Processingminus For problem diagnosis situation assessment etc in the

network

Distributed Problem Solvingminus Collaborative design scheduling and planning

23418 29

How to simulate SI for search

Example1 Ant --gt Ant Colony Optimization (ACO)

Example2 Bird Flocking --gt Particle Swarm Optimization (PSO)

23418 30

Part Ant Colony Optimization Ⅱ(ACO)

First proposed by M Dorigo 1992

23418 31

Natural Antsbull Individual ants are simple insects with

limited memory and capable of performing simple actions

bull However an ant colony expresses a complex collective behavior providing intelligent solutions to problems such asndash carrying large itemsndash forming bridgesndash finding the shortest routes from the nest

to a food source prioritizing food sources based on their distance and ease of access

23418 32

Natural Antsbull Moreover in a colony each ant has its prescribed task but

the ants can switch tasks if the collective needs it ndash Outside the nest ants can have 4 different tasks

bull Foraging searching for and retrieving food

bull Patrolling looking for food supply

bull Midden work Sorting the colony refuse pile

bull Nest maintenance work construction and clearing of chambers

ndash An antrsquos decision whether to perform a task depends onbull The Phisical State of the environment

ndash If part of the nest is damaged more ants do nest maintenance work to repair it

bull Social Interactions with other ants

Communication (direct or indirect) is necessary

23418 33

bull They establish indirect communication system based on the deposition of pheromone over the path they followndash An isolated ant moves at random but when it finds a pheromone

trail there is a high probability that this ant will decide to follow the trail

ndash An ant foraging for food deposits pheromone over its route When it finds a food source it returns to the nest reinforcing its trail

ndash So other ants have greater probability to start following this trail and thereby laying more pheromone on it

ndash This process works as a positive feedback loop system because the higher the intensity of the pheromone over a trail the higher the probability of an ant start traveling through it

How can the natural ants find the shortest path

23418 34

bull Since the route B is shorter the ants on this path will complete the travel more times and thereby lay more pheromone over it

bull The pheromone concentration on trail B will increase at a higher rate than on A and soon the ants on route A will choose to follow route B

bull Since most ants will no longer travel on route A and since the pheromone is volatile trail A will start evaporating

bull Only the shortest route will remain

23418 35

Problems of ASbull Ant System tends to converge quickly

ndash This means that its exploitation of the best solution found is too high it should be exploring solution space more

ndash Pheromone evaporationupdate rule (better rule may exist)

bull Led to extensions of the ant systemndash Elitist Strategy for Ant Systems (EAS)ndash Rank based Ant Systems (AKRANK)ndash MAX-MIN Ant system (MMAS)ndash Ant Colony System (ACS)

23418 36

Part Ⅲ Particle Swarm Optimization (PSO)

1048713Firstly Proposed by Kennedy and Eberhart 1995

ldquoOnce again nature has provided us with a technique for processing information that is at once elegant and versatilerdquo

23418 37

bull In PSO a ldquoswarmrdquo is defined as an apparently disorganized collection (population) of moving individuals that tend to cluster together while each individual seems to be moving in a random direction

Bird flocking is one of the best example of PSO in nature

23418 38

Modeling bird flocking

bull The synchrony of flocking behavior is thought to be a function of birdrsquos efforts to maintain an optimal distance between themselves and their neighborsndash Birds and fish adjust their physical movement to avoid

predators seek for food and mates ndash Humans tend to adjust our beliefs and attitudes to

conform with those of our social peers Humans change in abstract multidimensional space collision-free

23418 39

Modeling bird flocking (Contrsquod)

bull Definitions ndash Flock is a group of objects that exhibit the general

class of polarized (aligned) non-colliding aggregate motion

ndash Boid is a simulated bird-like object ie it exhibits this type of behavior It can be a fish bee dinosaur etc

bull Rules for flockingndash Cohesion Each boid fly towards the centroid of its

local flock mates (that is boid in its local neighborhood)ndash Separation Each boid keep a certain distance away

from local flock mates to avoid collisionsndash Alignment Each boid align its velocity vector and keep

velocity magnitude similar with that of the local flock

23418 40

bull Imagine a birdrsquos flock in an area where there is a single food source

bull A bird donrsquot know where the food is but it knows its distance to the food

bull The best strategy is to follow the bird that is closer to the food

bull Particles save and communicate the best solution they have found

From Bird to Particle

23418 41

Features of PSO

bull Population initialized by assigning random positions and velocities potential solutions are then flown through hyperspace

bull Each particle keeps track of its ldquobestrdquo (highest fitness) position in hyperspacendash This is called ldquopBestrdquo for an individual particlendash It is called ldquogBestrdquo for the best in the populationndash It is called ldquolBestrdquo for the best in a defined neighborhood

bull At each time step each particle stochastically accelerates toward its pBest and gBest (or lBest)

23418 42

Particle Swarm Optimization Process

Step1 Initialize population in hyperspace

Step2 Evaluate fitness of individual particles

Step3 Modify velocities based on previous best and global (or neighborhood) best

Step4 Terminate on some condition

Step5 Go to step 2

23418 43

How do particles fly

bull Combination of gBest and the pBest (lBest)ndash need a compromise

bull lBest can bendash Social the particles around are always the same no matter

where they are in spacendash Geographical the particles around are those whose distance is

the shortest

bull Global PSO vs Local PSOndash the global version converges quickly to a solution but it gets

more easily stuck in local minima

23418 44

Illustrating the velocity update schema of global PSO

23418 45

PSO Related issues

bull Controlling velocities (determining the best value for Vmax)ndash Usually set maximum velocity to dynamic range of variable

bull Usually set c1 and c2 to 2

bull Inertia weight influences tradeoff between global and local explorationndash Good approach is to reduce inertia weight during run (ie from

09 to 04 over 1000 generations)

bull Swarm Size and Neighborhood Size

23418 46

Advantages of PSO

bull Adaptation operates on velocitiesndash Most other methods operate on positionsndash Effect PSO has a builtin momentumndash Particles tend to hurdle past optima ndash an advantage

since the best positions are remembered anyway

bull Simple in conceptbull Easy to implementbull Computationally efficientbull Effective on a variety of problems

23418 47

Summarybull Swarm Intelligence (SI)

ndash an artificial intelligence technique based around the study of collective behavior in decentralized self-organized systems

bull Multi-Agent Systems (MAS)ndash A system that consists of a number of agents

which interact with one-anotherndash Communication Coordination

Collaboration

23418 48

bull Ant Colony Optimization (ACO)ndash Inspired by ant colony foragingndash pheromone as heuristic information

(stigmergy)ndash Iteration between ConstructAntSolutions and

UpdatePheromones

bull Particle Swarm Optimization (PSO)ndash Inspired by bird flockingndash Heuristic information results from partnersndash Particle Velocity Update

• Collective Intelligence
• Outline
• The Computational Beauty of Nature
• Examples of Collective Behavior in Nature and Society
• Emergence
• Biological motivation Insect Societies
• Insect Societies
• Self Organization
• Stigmergy
• Flocks Herds and Schools
• Boid rules
• Slide 12
• Swarm Intelligence (Contrsquod)
• Components of SI
• Characteristics of SI
• The Web becomes a Giant Brain Some see the Web evolving into a collective brain for humankind
• What is Multi-agent Systems
• What is MAS(Contrsquod)
• Slide 19
• Communication models
• Working together
• Example mechanism Contract Net Protocol (CNP)
• Slide 23
• CNP (Contrsquod)
• Slide 25
• Attributes of Multi-agent Systems
• Applications of MAS
• Applications of MAS (Contrsquod)
• How to simulate SI for search
• Part Ⅱ Ant Colony Optimization (ACO)
• Natural Ants
• Slide 32
• How can the natural ants find the shortest path
• Slide 34
• Problems of AS
• Part Ⅲ Particle Swarm Optimization (PSO)
• Slide 37
• Modeling bird flocking
• Modeling bird flocking (Contrsquod)
• From Bird to Particle
• Features of PSO
• Particle Swarm Optimization Process
• How do particles fly
• Slide 44
• PSO Related issues
• Advantages of PSO
• Summary
• Slide 48

23418 10

Flocks Herds and Schoolsbull In the late 80rsquos Craig Reynolds created a simple model of animal motion that he called Boids

bull Itrsquos generates very realistic motion for movement from three simple rules which define a boidrsquos steering behaviour

bull This model and its variations has been used to drive animations of birds insects people fish antelope etc in films (eg Batman Returns Lion King)

23418 11

Boid rulesSeparation steer to avoid crowding local flockmates

- A fundamental rule that has priority over the others- Also useful in avoiding collisions with other objects in the environment

Alignment steer towards the average heading and speed of local flockmates

- Enforces cohesion to keep the flock togetherHelps with collision avoidance too

Cohesion steer to move toward the average position of local flockmates

- Agents at edge of the herd more vulnerable to predators

- Helps to keep the flock together

23418 12

bull Swarm intelligence (SI) is an artificial intelligence technique based around the study of collective behavior in decentralized self-organized systems

bull The express lsquoSwarm Intelligencersquo was originally used by Beni Hackwood and Wang in 1989 in the context of cellular robotic systems to describe the self-organization of simple mechanical agents

bull It was later extended to include ldquoany attempt to design algorithms or distributed problem-solving devices inspired by the collective behavior of social insect colonies and other animal societiesrdquo [Bonabeau Dorigo and Theraulaz 1999]

Swarm Intelligence

23418 13

Swarm Intelligence (Contrsquod)

bull SI systems are typically made up of a population of simple agents interacting locally with one another and with their environment

bull Although there is normally no centralized control structure dictating how individual agents should behave local interactions between such agents often lead to the emergence of global behavior

bull Sometimes called lsquoCollective Intelligencersquo

23418 14

Components of SIbull Agents

ndash Interact with the world and with each other (either directly or indirectly)

bull Simple behavioursndash eg ants termites bees wasps

bull Communicationndash How agents interact with each otherndash eg pheromones of ants

Simple behaviours of individual agents

+ Communication between a group of agents

= Emergent complex behaviour of the group of agents

23418 15

Characteristics of SI

bull Distributed no central control or data source

bull Limited communication

bull No (explicit) model of the environment

bull Perception of environment (sensing)

bull Ability to react to environment changes

Is SI relevant to people

23418 16

The Web becomes a Giant Brain

Some see the Web evolving

intoa collective

brain for humankind

23418 17

What is Multi-agent Systemsbull A set of agents which interact in a common environmentbull Focus on the collaborative resolution of global

problems by a set of distributed entitiesbull Agent attempt to satisfy their own local goals as well as

the collaborative global goals bull To successfully interact they will require the ability to

cooperate coordinate and negotiate with each other much as people do

23418 18

What is MAS(Contrsquod)

bull MAS as seen from Distributed AIndash a loosely coupled network of entities that work

together to find answers to problems that are beyond the individual capabilities or knowledge of each entity

bull A more general meaningndash systems composed of autonomous components

that exhibit the following characteristicsbull each agent has incomplete capabilities to solve a problembull there is no global system controlbull data is decentralizedbull computation is asynchronous

23418 19

bull Traditionalndash Client-serverndash Low-level

messagesndash Synchronousndash Can not do the job

bull Agent breakthroughsndash Peer-to-peer

topologyndash Blackboard

coordination modelndash Encapsulated

messagingndash High-level message

protocols

Client ServerFunction(Parameters)

Return(Parameters)

Traditional Software

IntelligentAgents

IntelligentAgents

IntelligentAgents

Blackboard

MessageReply

Agents IntelligentAgentsIntelligent

Agents

IntelligentAgentsIntelligent

Agents

IntelligentAgentsIntelligent

Agents

23418 20

Communication models

bull Theoretical models Speech act theory

bull Practical modelsndash shared languages like KIF KQML DAMLndash service models like DAML-Sndash social convention protocols

23418 21

Working together

bull Benevolent Agentsndash assume agents are benevolent our best

interest is their best interest

bull Self-Interested Agentsndash Agents will be assumed to act to further their

own interests possibly at expense of othersndash Potential for conflict

23418 22

Example mechanism Contract Net Protocol (CNP)

bull Negotiation as a collaboration mechanism

bull Negotiation on how tasks should be shared

ndash A task (plan) may be decomposed in a hierarchy of subtasks (hierarchical planning)

ndash An agent may subcontract another agent to perform a (sub)task

Contract

Bid

agent agent

CNP

Task announceme

nt

23418 24

CNP (Contrsquod)

Contractor

Potential candidate agents

Task announcement (broadcast)

Contractor

Candidate Candidate

Bid

Bid

Phase 1 Task Announcement

- The contractor agent publicly announces a task

- Potential candidates evaluate the task according to their won skills and availability

Phase 2 Submission of Bids Proposals

- Agents that satisfy the requiremenst ie are able to perform the task send their bid proposal to the contractor

23418 25

CNP (Contrsquod)

Contractor

Selected candidate

Contractor

Contracted agent

Contract

Phase 3 Selection

- The selection of the best candidate is made by the contractor based on received bids and on the CVs of the candidates

Phase 4 Contract awarding

- A contract is established between the contractor and the selected candidate

- A privileged bilateral communication channel is established between the two agents

23418 26

Attributes of Multi-agent Systems

Apply MAS when some of the following features show up in a problem

bull Decentralizationbull Complex components often best described at

the knowledge levelbull Adaptive behaviorbull Complex interactionsbull Coordinationbull Emergent aggregate behaviors

23418 27

Applications of MAS

Advanced Manufacturing Management Systemsminus Agents as representatives of machines users business

processes etc

Intelligent Information Search on Internetminus Some agents may show learning capabilities (learn the

preferences of their users )

Intelligent security enforcement on Internetminus Agents are representative of sensors or IDSs

Shopping Agents in Electronic Commerceminus With search price comparison and bargaining capabilities

23418 28

Applications of MAS (Contrsquod)

Multi-agent auction in E-commerce

Distributed Surveillanceminus For information search or to look for special events informing

their users of relevant news

Distributed Signal Processingminus For problem diagnosis situation assessment etc in the

network

Distributed Problem Solvingminus Collaborative design scheduling and planning

23418 29

How to simulate SI for search

Example1 Ant --gt Ant Colony Optimization (ACO)

Example2 Bird Flocking --gt Particle Swarm Optimization (PSO)

23418 30

Part Ant Colony Optimization Ⅱ(ACO)

First proposed by M Dorigo 1992

23418 31

Natural Antsbull Individual ants are simple insects with

limited memory and capable of performing simple actions

bull However an ant colony expresses a complex collective behavior providing intelligent solutions to problems such asndash carrying large itemsndash forming bridgesndash finding the shortest routes from the nest

to a food source prioritizing food sources based on their distance and ease of access

23418 32

Natural Antsbull Moreover in a colony each ant has its prescribed task but

the ants can switch tasks if the collective needs it ndash Outside the nest ants can have 4 different tasks

bull Foraging searching for and retrieving food

bull Patrolling looking for food supply

bull Midden work Sorting the colony refuse pile

bull Nest maintenance work construction and clearing of chambers

ndash An antrsquos decision whether to perform a task depends onbull The Phisical State of the environment

ndash If part of the nest is damaged more ants do nest maintenance work to repair it

bull Social Interactions with other ants

Communication (direct or indirect) is necessary

23418 33

bull They establish indirect communication system based on the deposition of pheromone over the path they followndash An isolated ant moves at random but when it finds a pheromone

trail there is a high probability that this ant will decide to follow the trail

ndash An ant foraging for food deposits pheromone over its route When it finds a food source it returns to the nest reinforcing its trail

ndash So other ants have greater probability to start following this trail and thereby laying more pheromone on it

ndash This process works as a positive feedback loop system because the higher the intensity of the pheromone over a trail the higher the probability of an ant start traveling through it

How can the natural ants find the shortest path

23418 34

bull Since the route B is shorter the ants on this path will complete the travel more times and thereby lay more pheromone over it

bull The pheromone concentration on trail B will increase at a higher rate than on A and soon the ants on route A will choose to follow route B

bull Since most ants will no longer travel on route A and since the pheromone is volatile trail A will start evaporating

bull Only the shortest route will remain

23418 35

Problems of ASbull Ant System tends to converge quickly

ndash This means that its exploitation of the best solution found is too high it should be exploring solution space more

ndash Pheromone evaporationupdate rule (better rule may exist)

bull Led to extensions of the ant systemndash Elitist Strategy for Ant Systems (EAS)ndash Rank based Ant Systems (AKRANK)ndash MAX-MIN Ant system (MMAS)ndash Ant Colony System (ACS)

23418 36

Part Ⅲ Particle Swarm Optimization (PSO)

1048713Firstly Proposed by Kennedy and Eberhart 1995

ldquoOnce again nature has provided us with a technique for processing information that is at once elegant and versatilerdquo

23418 37

bull In PSO a ldquoswarmrdquo is defined as an apparently disorganized collection (population) of moving individuals that tend to cluster together while each individual seems to be moving in a random direction

Bird flocking is one of the best example of PSO in nature

23418 38

Modeling bird flocking

bull The synchrony of flocking behavior is thought to be a function of birdrsquos efforts to maintain an optimal distance between themselves and their neighborsndash Birds and fish adjust their physical movement to avoid

predators seek for food and mates ndash Humans tend to adjust our beliefs and attitudes to

conform with those of our social peers Humans change in abstract multidimensional space collision-free

23418 39

Modeling bird flocking (Contrsquod)

bull Definitions ndash Flock is a group of objects that exhibit the general

class of polarized (aligned) non-colliding aggregate motion

ndash Boid is a simulated bird-like object ie it exhibits this type of behavior It can be a fish bee dinosaur etc

bull Rules for flockingndash Cohesion Each boid fly towards the centroid of its

local flock mates (that is boid in its local neighborhood)ndash Separation Each boid keep a certain distance away

from local flock mates to avoid collisionsndash Alignment Each boid align its velocity vector and keep

velocity magnitude similar with that of the local flock

23418 40

bull Imagine a birdrsquos flock in an area where there is a single food source

bull A bird donrsquot know where the food is but it knows its distance to the food

bull The best strategy is to follow the bird that is closer to the food

bull Particles save and communicate the best solution they have found

From Bird to Particle

23418 41

Features of PSO

bull Population initialized by assigning random positions and velocities potential solutions are then flown through hyperspace

bull Each particle keeps track of its ldquobestrdquo (highest fitness) position in hyperspacendash This is called ldquopBestrdquo for an individual particlendash It is called ldquogBestrdquo for the best in the populationndash It is called ldquolBestrdquo for the best in a defined neighborhood

bull At each time step each particle stochastically accelerates toward its pBest and gBest (or lBest)

23418 42

Particle Swarm Optimization Process

Step1 Initialize population in hyperspace

Step2 Evaluate fitness of individual particles

Step3 Modify velocities based on previous best and global (or neighborhood) best

Step4 Terminate on some condition

Step5 Go to step 2

23418 43

How do particles fly

bull Combination of gBest and the pBest (lBest)ndash need a compromise

bull lBest can bendash Social the particles around are always the same no matter

where they are in spacendash Geographical the particles around are those whose distance is

the shortest

bull Global PSO vs Local PSOndash the global version converges quickly to a solution but it gets

more easily stuck in local minima

23418 44

Illustrating the velocity update schema of global PSO

23418 45

PSO Related issues

bull Controlling velocities (determining the best value for Vmax)ndash Usually set maximum velocity to dynamic range of variable

bull Usually set c1 and c2 to 2

bull Inertia weight influences tradeoff between global and local explorationndash Good approach is to reduce inertia weight during run (ie from

09 to 04 over 1000 generations)

bull Swarm Size and Neighborhood Size

23418 46

Advantages of PSO

bull Adaptation operates on velocitiesndash Most other methods operate on positionsndash Effect PSO has a builtin momentumndash Particles tend to hurdle past optima ndash an advantage

since the best positions are remembered anyway

bull Simple in conceptbull Easy to implementbull Computationally efficientbull Effective on a variety of problems

23418 47

Summarybull Swarm Intelligence (SI)

ndash an artificial intelligence technique based around the study of collective behavior in decentralized self-organized systems

bull Multi-Agent Systems (MAS)ndash A system that consists of a number of agents

which interact with one-anotherndash Communication Coordination

Collaboration

23418 48

bull Ant Colony Optimization (ACO)ndash Inspired by ant colony foragingndash pheromone as heuristic information

(stigmergy)ndash Iteration between ConstructAntSolutions and

UpdatePheromones

bull Particle Swarm Optimization (PSO)ndash Inspired by bird flockingndash Heuristic information results from partnersndash Particle Velocity Update

• Collective Intelligence
• Outline
• The Computational Beauty of Nature
• Examples of Collective Behavior in Nature and Society
• Emergence
• Biological motivation Insect Societies
• Insect Societies
• Self Organization
• Stigmergy
• Flocks Herds and Schools
• Boid rules
• Slide 12
• Swarm Intelligence (Contrsquod)
• Components of SI
• Characteristics of SI
• The Web becomes a Giant Brain Some see the Web evolving into a collective brain for humankind
• What is Multi-agent Systems
• What is MAS(Contrsquod)
• Slide 19
• Communication models
• Working together
• Example mechanism Contract Net Protocol (CNP)
• Slide 23
• CNP (Contrsquod)
• Slide 25
• Attributes of Multi-agent Systems
• Applications of MAS
• Applications of MAS (Contrsquod)
• How to simulate SI for search
• Part Ⅱ Ant Colony Optimization (ACO)
• Natural Ants
• Slide 32
• How can the natural ants find the shortest path
• Slide 34
• Problems of AS
• Part Ⅲ Particle Swarm Optimization (PSO)
• Slide 37
• Modeling bird flocking
• Modeling bird flocking (Contrsquod)
• From Bird to Particle
• Features of PSO
• Particle Swarm Optimization Process
• How do particles fly
• Slide 44
• PSO Related issues
• Advantages of PSO
• Summary
• Slide 48

23418 11

Boid rulesSeparation steer to avoid crowding local flockmates

- A fundamental rule that has priority over the others- Also useful in avoiding collisions with other objects in the environment

Alignment steer towards the average heading and speed of local flockmates

- Enforces cohesion to keep the flock togetherHelps with collision avoidance too

Cohesion steer to move toward the average position of local flockmates

- Agents at edge of the herd more vulnerable to predators

- Helps to keep the flock together

23418 12

bull Swarm intelligence (SI) is an artificial intelligence technique based around the study of collective behavior in decentralized self-organized systems

bull The express lsquoSwarm Intelligencersquo was originally used by Beni Hackwood and Wang in 1989 in the context of cellular robotic systems to describe the self-organization of simple mechanical agents

bull It was later extended to include ldquoany attempt to design algorithms or distributed problem-solving devices inspired by the collective behavior of social insect colonies and other animal societiesrdquo [Bonabeau Dorigo and Theraulaz 1999]

Swarm Intelligence

23418 13

Swarm Intelligence (Contrsquod)

bull SI systems are typically made up of a population of simple agents interacting locally with one another and with their environment

bull Although there is normally no centralized control structure dictating how individual agents should behave local interactions between such agents often lead to the emergence of global behavior

bull Sometimes called lsquoCollective Intelligencersquo

23418 14

Components of SIbull Agents

ndash Interact with the world and with each other (either directly or indirectly)

bull Simple behavioursndash eg ants termites bees wasps

bull Communicationndash How agents interact with each otherndash eg pheromones of ants

Simple behaviours of individual agents

+ Communication between a group of agents

= Emergent complex behaviour of the group of agents

23418 15

Characteristics of SI

bull Distributed no central control or data source

bull Limited communication

bull No (explicit) model of the environment

bull Perception of environment (sensing)

bull Ability to react to environment changes

Is SI relevant to people

23418 16

The Web becomes a Giant Brain

Some see the Web evolving

intoa collective

brain for humankind

23418 17

What is Multi-agent Systemsbull A set of agents which interact in a common environmentbull Focus on the collaborative resolution of global

problems by a set of distributed entitiesbull Agent attempt to satisfy their own local goals as well as

the collaborative global goals bull To successfully interact they will require the ability to

cooperate coordinate and negotiate with each other much as people do

23418 18

What is MAS(Contrsquod)

bull MAS as seen from Distributed AIndash a loosely coupled network of entities that work

together to find answers to problems that are beyond the individual capabilities or knowledge of each entity

bull A more general meaningndash systems composed of autonomous components

that exhibit the following characteristicsbull each agent has incomplete capabilities to solve a problembull there is no global system controlbull data is decentralizedbull computation is asynchronous

23418 19

bull Traditionalndash Client-serverndash Low-level

messagesndash Synchronousndash Can not do the job

bull Agent breakthroughsndash Peer-to-peer

topologyndash Blackboard

coordination modelndash Encapsulated

messagingndash High-level message

protocols

Client ServerFunction(Parameters)

Return(Parameters)

Traditional Software

IntelligentAgents

IntelligentAgents

IntelligentAgents

Blackboard

MessageReply

Agents IntelligentAgentsIntelligent

Agents

IntelligentAgentsIntelligent

Agents

IntelligentAgentsIntelligent

Agents

23418 20

Communication models

bull Theoretical models Speech act theory

bull Practical modelsndash shared languages like KIF KQML DAMLndash service models like DAML-Sndash social convention protocols

23418 21

Working together

bull Benevolent Agentsndash assume agents are benevolent our best

interest is their best interest

bull Self-Interested Agentsndash Agents will be assumed to act to further their

own interests possibly at expense of othersndash Potential for conflict

23418 22

Example mechanism Contract Net Protocol (CNP)

bull Negotiation as a collaboration mechanism

bull Negotiation on how tasks should be shared

ndash A task (plan) may be decomposed in a hierarchy of subtasks (hierarchical planning)

ndash An agent may subcontract another agent to perform a (sub)task

Contract

Bid

agent agent

CNP

Task announceme

nt

23418 24

CNP (Contrsquod)

Contractor

Potential candidate agents

Task announcement (broadcast)

Contractor

Candidate Candidate

Bid

Bid

Phase 1 Task Announcement

- The contractor agent publicly announces a task

- Potential candidates evaluate the task according to their won skills and availability

Phase 2 Submission of Bids Proposals

- Agents that satisfy the requiremenst ie are able to perform the task send their bid proposal to the contractor

23418 25

CNP (Contrsquod)

Contractor

Selected candidate

Contractor

Contracted agent

Contract

Phase 3 Selection

- The selection of the best candidate is made by the contractor based on received bids and on the CVs of the candidates

Phase 4 Contract awarding

- A contract is established between the contractor and the selected candidate

- A privileged bilateral communication channel is established between the two agents

23418 26

Attributes of Multi-agent Systems

Apply MAS when some of the following features show up in a problem

bull Decentralizationbull Complex components often best described at

the knowledge levelbull Adaptive behaviorbull Complex interactionsbull Coordinationbull Emergent aggregate behaviors

23418 27

Applications of MAS

Advanced Manufacturing Management Systemsminus Agents as representatives of machines users business

processes etc

Intelligent Information Search on Internetminus Some agents may show learning capabilities (learn the

preferences of their users )

Intelligent security enforcement on Internetminus Agents are representative of sensors or IDSs

Shopping Agents in Electronic Commerceminus With search price comparison and bargaining capabilities

23418 28

Applications of MAS (Contrsquod)

Multi-agent auction in E-commerce

Distributed Surveillanceminus For information search or to look for special events informing

their users of relevant news

Distributed Signal Processingminus For problem diagnosis situation assessment etc in the

network

Distributed Problem Solvingminus Collaborative design scheduling and planning

23418 29

How to simulate SI for search

Example1 Ant --gt Ant Colony Optimization (ACO)

Example2 Bird Flocking --gt Particle Swarm Optimization (PSO)

23418 30

Part Ant Colony Optimization Ⅱ(ACO)

First proposed by M Dorigo 1992

23418 31

Natural Antsbull Individual ants are simple insects with

limited memory and capable of performing simple actions

bull However an ant colony expresses a complex collective behavior providing intelligent solutions to problems such asndash carrying large itemsndash forming bridgesndash finding the shortest routes from the nest

to a food source prioritizing food sources based on their distance and ease of access

23418 32

Natural Antsbull Moreover in a colony each ant has its prescribed task but

the ants can switch tasks if the collective needs it ndash Outside the nest ants can have 4 different tasks

bull Foraging searching for and retrieving food

bull Patrolling looking for food supply

bull Midden work Sorting the colony refuse pile

bull Nest maintenance work construction and clearing of chambers

ndash An antrsquos decision whether to perform a task depends onbull The Phisical State of the environment

ndash If part of the nest is damaged more ants do nest maintenance work to repair it

bull Social Interactions with other ants

Communication (direct or indirect) is necessary

23418 33

bull They establish indirect communication system based on the deposition of pheromone over the path they followndash An isolated ant moves at random but when it finds a pheromone

trail there is a high probability that this ant will decide to follow the trail

ndash An ant foraging for food deposits pheromone over its route When it finds a food source it returns to the nest reinforcing its trail

ndash So other ants have greater probability to start following this trail and thereby laying more pheromone on it

ndash This process works as a positive feedback loop system because the higher the intensity of the pheromone over a trail the higher the probability of an ant start traveling through it

How can the natural ants find the shortest path

23418 34

bull Since the route B is shorter the ants on this path will complete the travel more times and thereby lay more pheromone over it

bull The pheromone concentration on trail B will increase at a higher rate than on A and soon the ants on route A will choose to follow route B

bull Since most ants will no longer travel on route A and since the pheromone is volatile trail A will start evaporating

bull Only the shortest route will remain

23418 35

Problems of ASbull Ant System tends to converge quickly

ndash This means that its exploitation of the best solution found is too high it should be exploring solution space more

ndash Pheromone evaporationupdate rule (better rule may exist)

bull Led to extensions of the ant systemndash Elitist Strategy for Ant Systems (EAS)ndash Rank based Ant Systems (AKRANK)ndash MAX-MIN Ant system (MMAS)ndash Ant Colony System (ACS)

23418 36

Part Ⅲ Particle Swarm Optimization (PSO)

1048713Firstly Proposed by Kennedy and Eberhart 1995

ldquoOnce again nature has provided us with a technique for processing information that is at once elegant and versatilerdquo

23418 37

bull In PSO a ldquoswarmrdquo is defined as an apparently disorganized collection (population) of moving individuals that tend to cluster together while each individual seems to be moving in a random direction

Bird flocking is one of the best example of PSO in nature

23418 38

Modeling bird flocking

bull The synchrony of flocking behavior is thought to be a function of birdrsquos efforts to maintain an optimal distance between themselves and their neighborsndash Birds and fish adjust their physical movement to avoid

predators seek for food and mates ndash Humans tend to adjust our beliefs and attitudes to

conform with those of our social peers Humans change in abstract multidimensional space collision-free

23418 39

Modeling bird flocking (Contrsquod)

bull Definitions ndash Flock is a group of objects that exhibit the general

class of polarized (aligned) non-colliding aggregate motion

ndash Boid is a simulated bird-like object ie it exhibits this type of behavior It can be a fish bee dinosaur etc

bull Rules for flockingndash Cohesion Each boid fly towards the centroid of its

local flock mates (that is boid in its local neighborhood)ndash Separation Each boid keep a certain distance away

from local flock mates to avoid collisionsndash Alignment Each boid align its velocity vector and keep

velocity magnitude similar with that of the local flock

23418 40

bull Imagine a birdrsquos flock in an area where there is a single food source

bull A bird donrsquot know where the food is but it knows its distance to the food

bull The best strategy is to follow the bird that is closer to the food

bull Particles save and communicate the best solution they have found

From Bird to Particle

23418 41

Features of PSO

bull Population initialized by assigning random positions and velocities potential solutions are then flown through hyperspace

bull Each particle keeps track of its ldquobestrdquo (highest fitness) position in hyperspacendash This is called ldquopBestrdquo for an individual particlendash It is called ldquogBestrdquo for the best in the populationndash It is called ldquolBestrdquo for the best in a defined neighborhood

bull At each time step each particle stochastically accelerates toward its pBest and gBest (or lBest)

23418 42

Particle Swarm Optimization Process

Step1 Initialize population in hyperspace

Step2 Evaluate fitness of individual particles

Step3 Modify velocities based on previous best and global (or neighborhood) best

Step4 Terminate on some condition

Step5 Go to step 2

23418 43

How do particles fly

bull Combination of gBest and the pBest (lBest)ndash need a compromise

bull lBest can bendash Social the particles around are always the same no matter

where they are in spacendash Geographical the particles around are those whose distance is

the shortest

bull Global PSO vs Local PSOndash the global version converges quickly to a solution but it gets

more easily stuck in local minima

23418 44

Illustrating the velocity update schema of global PSO

23418 45

PSO Related issues

bull Controlling velocities (determining the best value for Vmax)ndash Usually set maximum velocity to dynamic range of variable

bull Usually set c1 and c2 to 2

bull Inertia weight influences tradeoff between global and local explorationndash Good approach is to reduce inertia weight during run (ie from

09 to 04 over 1000 generations)

bull Swarm Size and Neighborhood Size

23418 46

Advantages of PSO

bull Adaptation operates on velocitiesndash Most other methods operate on positionsndash Effect PSO has a builtin momentumndash Particles tend to hurdle past optima ndash an advantage

since the best positions are remembered anyway

bull Simple in conceptbull Easy to implementbull Computationally efficientbull Effective on a variety of problems

23418 47

Summarybull Swarm Intelligence (SI)

ndash an artificial intelligence technique based around the study of collective behavior in decentralized self-organized systems

bull Multi-Agent Systems (MAS)ndash A system that consists of a number of agents

which interact with one-anotherndash Communication Coordination

Collaboration

23418 48

bull Ant Colony Optimization (ACO)ndash Inspired by ant colony foragingndash pheromone as heuristic information

(stigmergy)ndash Iteration between ConstructAntSolutions and

UpdatePheromones

bull Particle Swarm Optimization (PSO)ndash Inspired by bird flockingndash Heuristic information results from partnersndash Particle Velocity Update

• Collective Intelligence
• Outline
• The Computational Beauty of Nature
• Examples of Collective Behavior in Nature and Society
• Emergence
• Biological motivation Insect Societies
• Insect Societies
• Self Organization
• Stigmergy
• Flocks Herds and Schools
• Boid rules
• Slide 12
• Swarm Intelligence (Contrsquod)
• Components of SI
• Characteristics of SI
• The Web becomes a Giant Brain Some see the Web evolving into a collective brain for humankind
• What is Multi-agent Systems
• What is MAS(Contrsquod)
• Slide 19
• Communication models
• Working together
• Example mechanism Contract Net Protocol (CNP)
• Slide 23
• CNP (Contrsquod)
• Slide 25
• Attributes of Multi-agent Systems
• Applications of MAS
• Applications of MAS (Contrsquod)
• How to simulate SI for search
• Part Ⅱ Ant Colony Optimization (ACO)
• Natural Ants
• Slide 32
• How can the natural ants find the shortest path
• Slide 34
• Problems of AS
• Part Ⅲ Particle Swarm Optimization (PSO)
• Slide 37
• Modeling bird flocking
• Modeling bird flocking (Contrsquod)
• From Bird to Particle
• Features of PSO
• Particle Swarm Optimization Process
• How do particles fly
• Slide 44
• PSO Related issues
• Advantages of PSO
• Summary
• Slide 48

23418 12

bull Swarm intelligence (SI) is an artificial intelligence technique based around the study of collective behavior in decentralized self-organized systems

bull The express lsquoSwarm Intelligencersquo was originally used by Beni Hackwood and Wang in 1989 in the context of cellular robotic systems to describe the self-organization of simple mechanical agents

bull It was later extended to include ldquoany attempt to design algorithms or distributed problem-solving devices inspired by the collective behavior of social insect colonies and other animal societiesrdquo [Bonabeau Dorigo and Theraulaz 1999]

Swarm Intelligence

23418 13

Swarm Intelligence (Contrsquod)

bull SI systems are typically made up of a population of simple agents interacting locally with one another and with their environment

bull Although there is normally no centralized control structure dictating how individual agents should behave local interactions between such agents often lead to the emergence of global behavior

bull Sometimes called lsquoCollective Intelligencersquo

23418 14

Components of SIbull Agents

ndash Interact with the world and with each other (either directly or indirectly)

bull Simple behavioursndash eg ants termites bees wasps

bull Communicationndash How agents interact with each otherndash eg pheromones of ants

Simple behaviours of individual agents

+ Communication between a group of agents

= Emergent complex behaviour of the group of agents

23418 15

Characteristics of SI

bull Distributed no central control or data source

bull Limited communication

bull No (explicit) model of the environment

bull Perception of environment (sensing)

bull Ability to react to environment changes

Is SI relevant to people

23418 16

The Web becomes a Giant Brain

Some see the Web evolving

intoa collective

brain for humankind

23418 17

What is Multi-agent Systemsbull A set of agents which interact in a common environmentbull Focus on the collaborative resolution of global

problems by a set of distributed entitiesbull Agent attempt to satisfy their own local goals as well as

the collaborative global goals bull To successfully interact they will require the ability to

cooperate coordinate and negotiate with each other much as people do

23418 18

What is MAS(Contrsquod)

bull MAS as seen from Distributed AIndash a loosely coupled network of entities that work

together to find answers to problems that are beyond the individual capabilities or knowledge of each entity

bull A more general meaningndash systems composed of autonomous components

that exhibit the following characteristicsbull each agent has incomplete capabilities to solve a problembull there is no global system controlbull data is decentralizedbull computation is asynchronous

23418 19

bull Traditionalndash Client-serverndash Low-level

messagesndash Synchronousndash Can not do the job

bull Agent breakthroughsndash Peer-to-peer

topologyndash Blackboard

coordination modelndash Encapsulated

messagingndash High-level message

protocols

Client ServerFunction(Parameters)

Return(Parameters)

Traditional Software

IntelligentAgents

IntelligentAgents

IntelligentAgents

Blackboard

MessageReply

Agents IntelligentAgentsIntelligent

Agents

IntelligentAgentsIntelligent

Agents

IntelligentAgentsIntelligent

Agents

23418 20

Communication models

bull Theoretical models Speech act theory

bull Practical modelsndash shared languages like KIF KQML DAMLndash service models like DAML-Sndash social convention protocols

23418 21

Working together

bull Benevolent Agentsndash assume agents are benevolent our best

interest is their best interest

bull Self-Interested Agentsndash Agents will be assumed to act to further their

own interests possibly at expense of othersndash Potential for conflict

23418 22

Example mechanism Contract Net Protocol (CNP)

bull Negotiation as a collaboration mechanism

bull Negotiation on how tasks should be shared

ndash A task (plan) may be decomposed in a hierarchy of subtasks (hierarchical planning)

ndash An agent may subcontract another agent to perform a (sub)task

Contract

Bid

agent agent

CNP

Task announceme

nt

23418 24

CNP (Contrsquod)

Contractor

Potential candidate agents

Task announcement (broadcast)

Contractor

Candidate Candidate

Bid

Bid

Phase 1 Task Announcement

- The contractor agent publicly announces a task

- Potential candidates evaluate the task according to their won skills and availability

Phase 2 Submission of Bids Proposals

- Agents that satisfy the requiremenst ie are able to perform the task send their bid proposal to the contractor

23418 25

CNP (Contrsquod)

Contractor

Selected candidate

Contractor

Contracted agent

Contract

Phase 3 Selection

- The selection of the best candidate is made by the contractor based on received bids and on the CVs of the candidates

Phase 4 Contract awarding

- A contract is established between the contractor and the selected candidate

- A privileged bilateral communication channel is established between the two agents

23418 26

Attributes of Multi-agent Systems

Apply MAS when some of the following features show up in a problem

bull Decentralizationbull Complex components often best described at

the knowledge levelbull Adaptive behaviorbull Complex interactionsbull Coordinationbull Emergent aggregate behaviors

23418 27

Applications of MAS

Advanced Manufacturing Management Systemsminus Agents as representatives of machines users business

processes etc

Intelligent Information Search on Internetminus Some agents may show learning capabilities (learn the

preferences of their users )

Intelligent security enforcement on Internetminus Agents are representative of sensors or IDSs

Shopping Agents in Electronic Commerceminus With search price comparison and bargaining capabilities

23418 28

Applications of MAS (Contrsquod)

Multi-agent auction in E-commerce

Distributed Surveillanceminus For information search or to look for special events informing

their users of relevant news

Distributed Signal Processingminus For problem diagnosis situation assessment etc in the

network

Distributed Problem Solvingminus Collaborative design scheduling and planning

23418 29

How to simulate SI for search

Example1 Ant --gt Ant Colony Optimization (ACO)

Example2 Bird Flocking --gt Particle Swarm Optimization (PSO)

23418 30

Part Ant Colony Optimization Ⅱ(ACO)

First proposed by M Dorigo 1992

23418 31

Natural Antsbull Individual ants are simple insects with

limited memory and capable of performing simple actions

bull However an ant colony expresses a complex collective behavior providing intelligent solutions to problems such asndash carrying large itemsndash forming bridgesndash finding the shortest routes from the nest

to a food source prioritizing food sources based on their distance and ease of access

23418 32

Natural Antsbull Moreover in a colony each ant has its prescribed task but

the ants can switch tasks if the collective needs it ndash Outside the nest ants can have 4 different tasks

bull Foraging searching for and retrieving food

bull Patrolling looking for food supply

bull Midden work Sorting the colony refuse pile

bull Nest maintenance work construction and clearing of chambers

ndash An antrsquos decision whether to perform a task depends onbull The Phisical State of the environment

ndash If part of the nest is damaged more ants do nest maintenance work to repair it

bull Social Interactions with other ants

Communication (direct or indirect) is necessary

23418 33

bull They establish indirect communication system based on the deposition of pheromone over the path they followndash An isolated ant moves at random but when it finds a pheromone

trail there is a high probability that this ant will decide to follow the trail

ndash An ant foraging for food deposits pheromone over its route When it finds a food source it returns to the nest reinforcing its trail

ndash So other ants have greater probability to start following this trail and thereby laying more pheromone on it

ndash This process works as a positive feedback loop system because the higher the intensity of the pheromone over a trail the higher the probability of an ant start traveling through it

How can the natural ants find the shortest path

23418 34

bull Since the route B is shorter the ants on this path will complete the travel more times and thereby lay more pheromone over it

bull The pheromone concentration on trail B will increase at a higher rate than on A and soon the ants on route A will choose to follow route B

bull Since most ants will no longer travel on route A and since the pheromone is volatile trail A will start evaporating

bull Only the shortest route will remain

23418 35

Problems of ASbull Ant System tends to converge quickly

ndash This means that its exploitation of the best solution found is too high it should be exploring solution space more

ndash Pheromone evaporationupdate rule (better rule may exist)

bull Led to extensions of the ant systemndash Elitist Strategy for Ant Systems (EAS)ndash Rank based Ant Systems (AKRANK)ndash MAX-MIN Ant system (MMAS)ndash Ant Colony System (ACS)

23418 36

Part Ⅲ Particle Swarm Optimization (PSO)

1048713Firstly Proposed by Kennedy and Eberhart 1995

ldquoOnce again nature has provided us with a technique for processing information that is at once elegant and versatilerdquo

23418 37

bull In PSO a ldquoswarmrdquo is defined as an apparently disorganized collection (population) of moving individuals that tend to cluster together while each individual seems to be moving in a random direction

Bird flocking is one of the best example of PSO in nature

23418 38

Modeling bird flocking

bull The synchrony of flocking behavior is thought to be a function of birdrsquos efforts to maintain an optimal distance between themselves and their neighborsndash Birds and fish adjust their physical movement to avoid

predators seek for food and mates ndash Humans tend to adjust our beliefs and attitudes to

conform with those of our social peers Humans change in abstract multidimensional space collision-free

23418 39

Modeling bird flocking (Contrsquod)

bull Definitions ndash Flock is a group of objects that exhibit the general

class of polarized (aligned) non-colliding aggregate motion

ndash Boid is a simulated bird-like object ie it exhibits this type of behavior It can be a fish bee dinosaur etc

bull Rules for flockingndash Cohesion Each boid fly towards the centroid of its

local flock mates (that is boid in its local neighborhood)ndash Separation Each boid keep a certain distance away

from local flock mates to avoid collisionsndash Alignment Each boid align its velocity vector and keep

velocity magnitude similar with that of the local flock

23418 40

bull Imagine a birdrsquos flock in an area where there is a single food source

bull A bird donrsquot know where the food is but it knows its distance to the food

bull The best strategy is to follow the bird that is closer to the food

bull Particles save and communicate the best solution they have found

From Bird to Particle

23418 41

Features of PSO

bull Population initialized by assigning random positions and velocities potential solutions are then flown through hyperspace

bull Each particle keeps track of its ldquobestrdquo (highest fitness) position in hyperspacendash This is called ldquopBestrdquo for an individual particlendash It is called ldquogBestrdquo for the best in the populationndash It is called ldquolBestrdquo for the best in a defined neighborhood

bull At each time step each particle stochastically accelerates toward its pBest and gBest (or lBest)

23418 42

Particle Swarm Optimization Process

Step1 Initialize population in hyperspace

Step2 Evaluate fitness of individual particles

Step3 Modify velocities based on previous best and global (or neighborhood) best

Step4 Terminate on some condition

Step5 Go to step 2

23418 43

How do particles fly

bull Combination of gBest and the pBest (lBest)ndash need a compromise

bull lBest can bendash Social the particles around are always the same no matter

where they are in spacendash Geographical the particles around are those whose distance is

the shortest

bull Global PSO vs Local PSOndash the global version converges quickly to a solution but it gets

more easily stuck in local minima

23418 44

Illustrating the velocity update schema of global PSO

23418 45

PSO Related issues

bull Controlling velocities (determining the best value for Vmax)ndash Usually set maximum velocity to dynamic range of variable

bull Usually set c1 and c2 to 2

bull Inertia weight influences tradeoff between global and local explorationndash Good approach is to reduce inertia weight during run (ie from

09 to 04 over 1000 generations)

bull Swarm Size and Neighborhood Size

23418 46

Advantages of PSO

bull Adaptation operates on velocitiesndash Most other methods operate on positionsndash Effect PSO has a builtin momentumndash Particles tend to hurdle past optima ndash an advantage

since the best positions are remembered anyway

bull Simple in conceptbull Easy to implementbull Computationally efficientbull Effective on a variety of problems

23418 47

Summarybull Swarm Intelligence (SI)

ndash an artificial intelligence technique based around the study of collective behavior in decentralized self-organized systems

bull Multi-Agent Systems (MAS)ndash A system that consists of a number of agents

which interact with one-anotherndash Communication Coordination

Collaboration

23418 48

bull Ant Colony Optimization (ACO)ndash Inspired by ant colony foragingndash pheromone as heuristic information

(stigmergy)ndash Iteration between ConstructAntSolutions and

UpdatePheromones

bull Particle Swarm Optimization (PSO)ndash Inspired by bird flockingndash Heuristic information results from partnersndash Particle Velocity Update

• Collective Intelligence
• Outline
• The Computational Beauty of Nature
• Examples of Collective Behavior in Nature and Society
• Emergence
• Biological motivation Insect Societies
• Insect Societies
• Self Organization
• Stigmergy
• Flocks Herds and Schools
• Boid rules
• Slide 12
• Swarm Intelligence (Contrsquod)
• Components of SI
• Characteristics of SI
• The Web becomes a Giant Brain Some see the Web evolving into a collective brain for humankind
• What is Multi-agent Systems
• What is MAS(Contrsquod)
• Slide 19
• Communication models
• Working together
• Example mechanism Contract Net Protocol (CNP)
• Slide 23
• CNP (Contrsquod)
• Slide 25
• Attributes of Multi-agent Systems
• Applications of MAS
• Applications of MAS (Contrsquod)
• How to simulate SI for search
• Part Ⅱ Ant Colony Optimization (ACO)
• Natural Ants
• Slide 32
• How can the natural ants find the shortest path
• Slide 34
• Problems of AS
• Part Ⅲ Particle Swarm Optimization (PSO)
• Slide 37
• Modeling bird flocking
• Modeling bird flocking (Contrsquod)
• From Bird to Particle
• Features of PSO
• Particle Swarm Optimization Process
• How do particles fly
• Slide 44
• PSO Related issues
• Advantages of PSO
• Summary
• Slide 48

23418 13

Swarm Intelligence (Contrsquod)

bull SI systems are typically made up of a population of simple agents interacting locally with one another and with their environment

bull Although there is normally no centralized control structure dictating how individual agents should behave local interactions between such agents often lead to the emergence of global behavior

bull Sometimes called lsquoCollective Intelligencersquo

23418 14

Components of SIbull Agents

ndash Interact with the world and with each other (either directly or indirectly)

bull Simple behavioursndash eg ants termites bees wasps

bull Communicationndash How agents interact with each otherndash eg pheromones of ants

Simple behaviours of individual agents

+ Communication between a group of agents

= Emergent complex behaviour of the group of agents

23418 15

Characteristics of SI

bull Distributed no central control or data source

bull Limited communication

bull No (explicit) model of the environment

bull Perception of environment (sensing)

bull Ability to react to environment changes

Is SI relevant to people

23418 16

The Web becomes a Giant Brain

Some see the Web evolving

intoa collective

brain for humankind

23418 17

What is Multi-agent Systemsbull A set of agents which interact in a common environmentbull Focus on the collaborative resolution of global

problems by a set of distributed entitiesbull Agent attempt to satisfy their own local goals as well as

the collaborative global goals bull To successfully interact they will require the ability to

cooperate coordinate and negotiate with each other much as people do

23418 18

What is MAS(Contrsquod)

bull MAS as seen from Distributed AIndash a loosely coupled network of entities that work

together to find answers to problems that are beyond the individual capabilities or knowledge of each entity

bull A more general meaningndash systems composed of autonomous components

that exhibit the following characteristicsbull each agent has incomplete capabilities to solve a problembull there is no global system controlbull data is decentralizedbull computation is asynchronous

23418 19

bull Traditionalndash Client-serverndash Low-level

messagesndash Synchronousndash Can not do the job

bull Agent breakthroughsndash Peer-to-peer

topologyndash Blackboard

coordination modelndash Encapsulated

messagingndash High-level message

protocols

Client ServerFunction(Parameters)

Return(Parameters)

Traditional Software

IntelligentAgents

IntelligentAgents

IntelligentAgents

Blackboard

MessageReply

Agents IntelligentAgentsIntelligent

Agents

IntelligentAgentsIntelligent

Agents

IntelligentAgentsIntelligent

Agents

23418 20

Communication models

bull Theoretical models Speech act theory

bull Practical modelsndash shared languages like KIF KQML DAMLndash service models like DAML-Sndash social convention protocols

23418 21

Working together

bull Benevolent Agentsndash assume agents are benevolent our best

interest is their best interest

bull Self-Interested Agentsndash Agents will be assumed to act to further their

own interests possibly at expense of othersndash Potential for conflict

23418 22

Example mechanism Contract Net Protocol (CNP)

bull Negotiation as a collaboration mechanism

bull Negotiation on how tasks should be shared

ndash A task (plan) may be decomposed in a hierarchy of subtasks (hierarchical planning)

ndash An agent may subcontract another agent to perform a (sub)task

Contract

Bid

agent agent

CNP

Task announceme

nt

23418 24

CNP (Contrsquod)

Contractor

Potential candidate agents

Task announcement (broadcast)

Contractor

Candidate Candidate

Bid

Bid

Phase 1 Task Announcement

- The contractor agent publicly announces a task

- Potential candidates evaluate the task according to their won skills and availability

Phase 2 Submission of Bids Proposals

- Agents that satisfy the requiremenst ie are able to perform the task send their bid proposal to the contractor

23418 25

CNP (Contrsquod)

Contractor

Selected candidate

Contractor

Contracted agent

Contract

Phase 3 Selection

- The selection of the best candidate is made by the contractor based on received bids and on the CVs of the candidates

Phase 4 Contract awarding

- A contract is established between the contractor and the selected candidate

- A privileged bilateral communication channel is established between the two agents

23418 26

Attributes of Multi-agent Systems

Apply MAS when some of the following features show up in a problem

bull Decentralizationbull Complex components often best described at

the knowledge levelbull Adaptive behaviorbull Complex interactionsbull Coordinationbull Emergent aggregate behaviors

23418 27

Applications of MAS

Advanced Manufacturing Management Systemsminus Agents as representatives of machines users business

processes etc

Intelligent Information Search on Internetminus Some agents may show learning capabilities (learn the

preferences of their users )

Intelligent security enforcement on Internetminus Agents are representative of sensors or IDSs

Shopping Agents in Electronic Commerceminus With search price comparison and bargaining capabilities

23418 28

Applications of MAS (Contrsquod)

Multi-agent auction in E-commerce

Distributed Surveillanceminus For information search or to look for special events informing

their users of relevant news

Distributed Signal Processingminus For problem diagnosis situation assessment etc in the

network

Distributed Problem Solvingminus Collaborative design scheduling and planning

23418 29

How to simulate SI for search

Example1 Ant --gt Ant Colony Optimization (ACO)

Example2 Bird Flocking --gt Particle Swarm Optimization (PSO)

23418 30

Part Ant Colony Optimization Ⅱ(ACO)

First proposed by M Dorigo 1992

23418 31

Natural Antsbull Individual ants are simple insects with

limited memory and capable of performing simple actions

bull However an ant colony expresses a complex collective behavior providing intelligent solutions to problems such asndash carrying large itemsndash forming bridgesndash finding the shortest routes from the nest

to a food source prioritizing food sources based on their distance and ease of access

23418 32

Natural Antsbull Moreover in a colony each ant has its prescribed task but

the ants can switch tasks if the collective needs it ndash Outside the nest ants can have 4 different tasks

bull Foraging searching for and retrieving food

bull Patrolling looking for food supply

bull Midden work Sorting the colony refuse pile

bull Nest maintenance work construction and clearing of chambers

ndash An antrsquos decision whether to perform a task depends onbull The Phisical State of the environment

ndash If part of the nest is damaged more ants do nest maintenance work to repair it

bull Social Interactions with other ants

Communication (direct or indirect) is necessary

23418 33

bull They establish indirect communication system based on the deposition of pheromone over the path they followndash An isolated ant moves at random but when it finds a pheromone

trail there is a high probability that this ant will decide to follow the trail

ndash An ant foraging for food deposits pheromone over its route When it finds a food source it returns to the nest reinforcing its trail

ndash So other ants have greater probability to start following this trail and thereby laying more pheromone on it

ndash This process works as a positive feedback loop system because the higher the intensity of the pheromone over a trail the higher the probability of an ant start traveling through it

How can the natural ants find the shortest path

23418 34

bull Since the route B is shorter the ants on this path will complete the travel more times and thereby lay more pheromone over it

bull The pheromone concentration on trail B will increase at a higher rate than on A and soon the ants on route A will choose to follow route B

bull Since most ants will no longer travel on route A and since the pheromone is volatile trail A will start evaporating

bull Only the shortest route will remain

23418 35

Problems of ASbull Ant System tends to converge quickly

ndash This means that its exploitation of the best solution found is too high it should be exploring solution space more

ndash Pheromone evaporationupdate rule (better rule may exist)

bull Led to extensions of the ant systemndash Elitist Strategy for Ant Systems (EAS)ndash Rank based Ant Systems (AKRANK)ndash MAX-MIN Ant system (MMAS)ndash Ant Colony System (ACS)

23418 36

Part Ⅲ Particle Swarm Optimization (PSO)

1048713Firstly Proposed by Kennedy and Eberhart 1995

ldquoOnce again nature has provided us with a technique for processing information that is at once elegant and versatilerdquo

23418 37

bull In PSO a ldquoswarmrdquo is defined as an apparently disorganized collection (population) of moving individuals that tend to cluster together while each individual seems to be moving in a random direction

Bird flocking is one of the best example of PSO in nature

23418 38

Modeling bird flocking

bull The synchrony of flocking behavior is thought to be a function of birdrsquos efforts to maintain an optimal distance between themselves and their neighborsndash Birds and fish adjust their physical movement to avoid

predators seek for food and mates ndash Humans tend to adjust our beliefs and attitudes to

conform with those of our social peers Humans change in abstract multidimensional space collision-free

23418 39

Modeling bird flocking (Contrsquod)

bull Definitions ndash Flock is a group of objects that exhibit the general

class of polarized (aligned) non-colliding aggregate motion

ndash Boid is a simulated bird-like object ie it exhibits this type of behavior It can be a fish bee dinosaur etc

bull Rules for flockingndash Cohesion Each boid fly towards the centroid of its

local flock mates (that is boid in its local neighborhood)ndash Separation Each boid keep a certain distance away

from local flock mates to avoid collisionsndash Alignment Each boid align its velocity vector and keep

velocity magnitude similar with that of the local flock

23418 40

bull Imagine a birdrsquos flock in an area where there is a single food source

bull A bird donrsquot know where the food is but it knows its distance to the food

bull The best strategy is to follow the bird that is closer to the food

bull Particles save and communicate the best solution they have found

From Bird to Particle

23418 41

Features of PSO

bull Population initialized by assigning random positions and velocities potential solutions are then flown through hyperspace

bull Each particle keeps track of its ldquobestrdquo (highest fitness) position in hyperspacendash This is called ldquopBestrdquo for an individual particlendash It is called ldquogBestrdquo for the best in the populationndash It is called ldquolBestrdquo for the best in a defined neighborhood

bull At each time step each particle stochastically accelerates toward its pBest and gBest (or lBest)

23418 42

Particle Swarm Optimization Process

Step1 Initialize population in hyperspace

Step2 Evaluate fitness of individual particles

Step3 Modify velocities based on previous best and global (or neighborhood) best

Step4 Terminate on some condition

Step5 Go to step 2

23418 43

How do particles fly

bull Combination of gBest and the pBest (lBest)ndash need a compromise

bull lBest can bendash Social the particles around are always the same no matter

where they are in spacendash Geographical the particles around are those whose distance is

the shortest

bull Global PSO vs Local PSOndash the global version converges quickly to a solution but it gets

more easily stuck in local minima

23418 44

Illustrating the velocity update schema of global PSO

23418 45

PSO Related issues

bull Controlling velocities (determining the best value for Vmax)ndash Usually set maximum velocity to dynamic range of variable

bull Usually set c1 and c2 to 2

bull Inertia weight influences tradeoff between global and local explorationndash Good approach is to reduce inertia weight during run (ie from

09 to 04 over 1000 generations)

bull Swarm Size and Neighborhood Size

23418 46

Advantages of PSO

bull Adaptation operates on velocitiesndash Most other methods operate on positionsndash Effect PSO has a builtin momentumndash Particles tend to hurdle past optima ndash an advantage

since the best positions are remembered anyway

bull Simple in conceptbull Easy to implementbull Computationally efficientbull Effective on a variety of problems

23418 47

Summarybull Swarm Intelligence (SI)

ndash an artificial intelligence technique based around the study of collective behavior in decentralized self-organized systems

bull Multi-Agent Systems (MAS)ndash A system that consists of a number of agents

which interact with one-anotherndash Communication Coordination

Collaboration

23418 48

bull Ant Colony Optimization (ACO)ndash Inspired by ant colony foragingndash pheromone as heuristic information

(stigmergy)ndash Iteration between ConstructAntSolutions and

UpdatePheromones

bull Particle Swarm Optimization (PSO)ndash Inspired by bird flockingndash Heuristic information results from partnersndash Particle Velocity Update

• Collective Intelligence
• Outline
• The Computational Beauty of Nature
• Examples of Collective Behavior in Nature and Society
• Emergence
• Biological motivation Insect Societies
• Insect Societies
• Self Organization
• Stigmergy
• Flocks Herds and Schools
• Boid rules
• Slide 12
• Swarm Intelligence (Contrsquod)
• Components of SI
• Characteristics of SI
• The Web becomes a Giant Brain Some see the Web evolving into a collective brain for humankind
• What is Multi-agent Systems
• What is MAS(Contrsquod)
• Slide 19
• Communication models
• Working together
• Example mechanism Contract Net Protocol (CNP)
• Slide 23
• CNP (Contrsquod)
• Slide 25
• Attributes of Multi-agent Systems
• Applications of MAS
• Applications of MAS (Contrsquod)
• How to simulate SI for search
• Part Ⅱ Ant Colony Optimization (ACO)
• Natural Ants
• Slide 32
• How can the natural ants find the shortest path
• Slide 34
• Problems of AS
• Part Ⅲ Particle Swarm Optimization (PSO)
• Slide 37
• Modeling bird flocking
• Modeling bird flocking (Contrsquod)
• From Bird to Particle
• Features of PSO
• Particle Swarm Optimization Process
• How do particles fly
• Slide 44
• PSO Related issues
• Advantages of PSO
• Summary
• Slide 48

23418 14

Components of SIbull Agents

ndash Interact with the world and with each other (either directly or indirectly)

bull Simple behavioursndash eg ants termites bees wasps

bull Communicationndash How agents interact with each otherndash eg pheromones of ants

Simple behaviours of individual agents

+ Communication between a group of agents

= Emergent complex behaviour of the group of agents

23418 15

Characteristics of SI

bull Distributed no central control or data source

bull Limited communication

bull No (explicit) model of the environment

bull Perception of environment (sensing)

bull Ability to react to environment changes

Is SI relevant to people

23418 16

The Web becomes a Giant Brain

Some see the Web evolving

intoa collective

brain for humankind

23418 17

What is Multi-agent Systemsbull A set of agents which interact in a common environmentbull Focus on the collaborative resolution of global

problems by a set of distributed entitiesbull Agent attempt to satisfy their own local goals as well as

the collaborative global goals bull To successfully interact they will require the ability to

cooperate coordinate and negotiate with each other much as people do

23418 18

What is MAS(Contrsquod)

bull MAS as seen from Distributed AIndash a loosely coupled network of entities that work

together to find answers to problems that are beyond the individual capabilities or knowledge of each entity

bull A more general meaningndash systems composed of autonomous components

that exhibit the following characteristicsbull each agent has incomplete capabilities to solve a problembull there is no global system controlbull data is decentralizedbull computation is asynchronous

23418 19

bull Traditionalndash Client-serverndash Low-level

messagesndash Synchronousndash Can not do the job

bull Agent breakthroughsndash Peer-to-peer

topologyndash Blackboard

coordination modelndash Encapsulated

messagingndash High-level message

protocols

Client ServerFunction(Parameters)

Return(Parameters)

Traditional Software

IntelligentAgents

IntelligentAgents

IntelligentAgents

Blackboard

MessageReply

Agents IntelligentAgentsIntelligent

Agents

IntelligentAgentsIntelligent

Agents

IntelligentAgentsIntelligent

Agents

23418 20

Communication models

bull Theoretical models Speech act theory

bull Practical modelsndash shared languages like KIF KQML DAMLndash service models like DAML-Sndash social convention protocols

23418 21

Working together

bull Benevolent Agentsndash assume agents are benevolent our best

interest is their best interest

bull Self-Interested Agentsndash Agents will be assumed to act to further their

own interests possibly at expense of othersndash Potential for conflict

23418 22

Example mechanism Contract Net Protocol (CNP)

bull Negotiation as a collaboration mechanism

bull Negotiation on how tasks should be shared

ndash A task (plan) may be decomposed in a hierarchy of subtasks (hierarchical planning)

ndash An agent may subcontract another agent to perform a (sub)task

Contract

Bid

agent agent

CNP

Task announceme

nt

23418 24

CNP (Contrsquod)

Contractor

Potential candidate agents

Task announcement (broadcast)

Contractor

Candidate Candidate

Bid

Bid

Phase 1 Task Announcement

- The contractor agent publicly announces a task

- Potential candidates evaluate the task according to their won skills and availability

Phase 2 Submission of Bids Proposals

- Agents that satisfy the requiremenst ie are able to perform the task send their bid proposal to the contractor

23418 25

CNP (Contrsquod)

Contractor

Selected candidate

Contractor

Contracted agent

Contract

Phase 3 Selection

- The selection of the best candidate is made by the contractor based on received bids and on the CVs of the candidates

Phase 4 Contract awarding

- A contract is established between the contractor and the selected candidate

- A privileged bilateral communication channel is established between the two agents

23418 26

Attributes of Multi-agent Systems

Apply MAS when some of the following features show up in a problem

bull Decentralizationbull Complex components often best described at

the knowledge levelbull Adaptive behaviorbull Complex interactionsbull Coordinationbull Emergent aggregate behaviors

23418 27

Applications of MAS

Advanced Manufacturing Management Systemsminus Agents as representatives of machines users business

processes etc

Intelligent Information Search on Internetminus Some agents may show learning capabilities (learn the

preferences of their users )

Intelligent security enforcement on Internetminus Agents are representative of sensors or IDSs

Shopping Agents in Electronic Commerceminus With search price comparison and bargaining capabilities

23418 28

Applications of MAS (Contrsquod)

Multi-agent auction in E-commerce

Distributed Surveillanceminus For information search or to look for special events informing

their users of relevant news

Distributed Signal Processingminus For problem diagnosis situation assessment etc in the

network

Distributed Problem Solvingminus Collaborative design scheduling and planning

23418 29

How to simulate SI for search

Example1 Ant --gt Ant Colony Optimization (ACO)

Example2 Bird Flocking --gt Particle Swarm Optimization (PSO)

23418 30

Part Ant Colony Optimization Ⅱ(ACO)

First proposed by M Dorigo 1992

23418 31

Natural Antsbull Individual ants are simple insects with

limited memory and capable of performing simple actions

bull However an ant colony expresses a complex collective behavior providing intelligent solutions to problems such asndash carrying large itemsndash forming bridgesndash finding the shortest routes from the nest

to a food source prioritizing food sources based on their distance and ease of access

23418 32

Natural Antsbull Moreover in a colony each ant has its prescribed task but

the ants can switch tasks if the collective needs it ndash Outside the nest ants can have 4 different tasks

bull Foraging searching for and retrieving food

bull Patrolling looking for food supply

bull Midden work Sorting the colony refuse pile

bull Nest maintenance work construction and clearing of chambers

ndash An antrsquos decision whether to perform a task depends onbull The Phisical State of the environment

ndash If part of the nest is damaged more ants do nest maintenance work to repair it

bull Social Interactions with other ants

Communication (direct or indirect) is necessary

23418 33

bull They establish indirect communication system based on the deposition of pheromone over the path they followndash An isolated ant moves at random but when it finds a pheromone

trail there is a high probability that this ant will decide to follow the trail

ndash An ant foraging for food deposits pheromone over its route When it finds a food source it returns to the nest reinforcing its trail

ndash So other ants have greater probability to start following this trail and thereby laying more pheromone on it

ndash This process works as a positive feedback loop system because the higher the intensity of the pheromone over a trail the higher the probability of an ant start traveling through it

How can the natural ants find the shortest path

23418 34

bull Since the route B is shorter the ants on this path will complete the travel more times and thereby lay more pheromone over it

bull The pheromone concentration on trail B will increase at a higher rate than on A and soon the ants on route A will choose to follow route B

bull Since most ants will no longer travel on route A and since the pheromone is volatile trail A will start evaporating

bull Only the shortest route will remain

23418 35

Problems of ASbull Ant System tends to converge quickly

ndash This means that its exploitation of the best solution found is too high it should be exploring solution space more

ndash Pheromone evaporationupdate rule (better rule may exist)

bull Led to extensions of the ant systemndash Elitist Strategy for Ant Systems (EAS)ndash Rank based Ant Systems (AKRANK)ndash MAX-MIN Ant system (MMAS)ndash Ant Colony System (ACS)

23418 36

Part Ⅲ Particle Swarm Optimization (PSO)

1048713Firstly Proposed by Kennedy and Eberhart 1995

ldquoOnce again nature has provided us with a technique for processing information that is at once elegant and versatilerdquo

23418 37

bull In PSO a ldquoswarmrdquo is defined as an apparently disorganized collection (population) of moving individuals that tend to cluster together while each individual seems to be moving in a random direction

Bird flocking is one of the best example of PSO in nature

23418 38

Modeling bird flocking

bull The synchrony of flocking behavior is thought to be a function of birdrsquos efforts to maintain an optimal distance between themselves and their neighborsndash Birds and fish adjust their physical movement to avoid

predators seek for food and mates ndash Humans tend to adjust our beliefs and attitudes to

conform with those of our social peers Humans change in abstract multidimensional space collision-free

23418 39

Modeling bird flocking (Contrsquod)

bull Definitions ndash Flock is a group of objects that exhibit the general

class of polarized (aligned) non-colliding aggregate motion

ndash Boid is a simulated bird-like object ie it exhibits this type of behavior It can be a fish bee dinosaur etc

bull Rules for flockingndash Cohesion Each boid fly towards the centroid of its

local flock mates (that is boid in its local neighborhood)ndash Separation Each boid keep a certain distance away

from local flock mates to avoid collisionsndash Alignment Each boid align its velocity vector and keep

velocity magnitude similar with that of the local flock

23418 40

bull Imagine a birdrsquos flock in an area where there is a single food source

bull A bird donrsquot know where the food is but it knows its distance to the food

bull The best strategy is to follow the bird that is closer to the food

bull Particles save and communicate the best solution they have found

From Bird to Particle

23418 41

Features of PSO

bull Population initialized by assigning random positions and velocities potential solutions are then flown through hyperspace

bull Each particle keeps track of its ldquobestrdquo (highest fitness) position in hyperspacendash This is called ldquopBestrdquo for an individual particlendash It is called ldquogBestrdquo for the best in the populationndash It is called ldquolBestrdquo for the best in a defined neighborhood

bull At each time step each particle stochastically accelerates toward its pBest and gBest (or lBest)

23418 42

Particle Swarm Optimization Process

Step1 Initialize population in hyperspace

Step2 Evaluate fitness of individual particles

Step3 Modify velocities based on previous best and global (or neighborhood) best

Step4 Terminate on some condition

Step5 Go to step 2

23418 43

How do particles fly

bull Combination of gBest and the pBest (lBest)ndash need a compromise

bull lBest can bendash Social the particles around are always the same no matter

where they are in spacendash Geographical the particles around are those whose distance is

the shortest

bull Global PSO vs Local PSOndash the global version converges quickly to a solution but it gets

more easily stuck in local minima

23418 44

Illustrating the velocity update schema of global PSO

23418 45

PSO Related issues

bull Controlling velocities (determining the best value for Vmax)ndash Usually set maximum velocity to dynamic range of variable

bull Usually set c1 and c2 to 2

bull Inertia weight influences tradeoff between global and local explorationndash Good approach is to reduce inertia weight during run (ie from

09 to 04 over 1000 generations)

bull Swarm Size and Neighborhood Size

23418 46

Advantages of PSO

bull Adaptation operates on velocitiesndash Most other methods operate on positionsndash Effect PSO has a builtin momentumndash Particles tend to hurdle past optima ndash an advantage

since the best positions are remembered anyway

bull Simple in conceptbull Easy to implementbull Computationally efficientbull Effective on a variety of problems

23418 47

Summarybull Swarm Intelligence (SI)

ndash an artificial intelligence technique based around the study of collective behavior in decentralized self-organized systems

bull Multi-Agent Systems (MAS)ndash A system that consists of a number of agents

which interact with one-anotherndash Communication Coordination

Collaboration

23418 48

bull Ant Colony Optimization (ACO)ndash Inspired by ant colony foragingndash pheromone as heuristic information

(stigmergy)ndash Iteration between ConstructAntSolutions and

UpdatePheromones

bull Particle Swarm Optimization (PSO)ndash Inspired by bird flockingndash Heuristic information results from partnersndash Particle Velocity Update

• Collective Intelligence
• Outline
• The Computational Beauty of Nature
• Examples of Collective Behavior in Nature and Society
• Emergence
• Biological motivation Insect Societies
• Insect Societies
• Self Organization
• Stigmergy
• Flocks Herds and Schools
• Boid rules
• Slide 12
• Swarm Intelligence (Contrsquod)
• Components of SI
• Characteristics of SI
• The Web becomes a Giant Brain Some see the Web evolving into a collective brain for humankind
• What is Multi-agent Systems
• What is MAS(Contrsquod)
• Slide 19
• Communication models
• Working together
• Example mechanism Contract Net Protocol (CNP)
• Slide 23
• CNP (Contrsquod)
• Slide 25
• Attributes of Multi-agent Systems
• Applications of MAS
• Applications of MAS (Contrsquod)
• How to simulate SI for search
• Part Ⅱ Ant Colony Optimization (ACO)
• Natural Ants
• Slide 32
• How can the natural ants find the shortest path
• Slide 34
• Problems of AS
• Part Ⅲ Particle Swarm Optimization (PSO)
• Slide 37
• Modeling bird flocking
• Modeling bird flocking (Contrsquod)
• From Bird to Particle
• Features of PSO
• Particle Swarm Optimization Process
• How do particles fly
• Slide 44
• PSO Related issues
• Advantages of PSO
• Summary
• Slide 48

23418 15

Characteristics of SI

bull Distributed no central control or data source

bull Limited communication

bull No (explicit) model of the environment

bull Perception of environment (sensing)

bull Ability to react to environment changes

Is SI relevant to people

23418 16

The Web becomes a Giant Brain

Some see the Web evolving

intoa collective

brain for humankind

23418 17

What is Multi-agent Systemsbull A set of agents which interact in a common environmentbull Focus on the collaborative resolution of global

problems by a set of distributed entitiesbull Agent attempt to satisfy their own local goals as well as

the collaborative global goals bull To successfully interact they will require the ability to

cooperate coordinate and negotiate with each other much as people do

23418 18

What is MAS(Contrsquod)

bull MAS as seen from Distributed AIndash a loosely coupled network of entities that work

together to find answers to problems that are beyond the individual capabilities or knowledge of each entity

bull A more general meaningndash systems composed of autonomous components

that exhibit the following characteristicsbull each agent has incomplete capabilities to solve a problembull there is no global system controlbull data is decentralizedbull computation is asynchronous

23418 19

bull Traditionalndash Client-serverndash Low-level

messagesndash Synchronousndash Can not do the job

bull Agent breakthroughsndash Peer-to-peer

topologyndash Blackboard

coordination modelndash Encapsulated

messagingndash High-level message

protocols

Client ServerFunction(Parameters)

Return(Parameters)

Traditional Software

IntelligentAgents

IntelligentAgents

IntelligentAgents

Blackboard

MessageReply

Agents IntelligentAgentsIntelligent

Agents

IntelligentAgentsIntelligent

Agents

IntelligentAgentsIntelligent

Agents

23418 20

Communication models

bull Theoretical models Speech act theory

bull Practical modelsndash shared languages like KIF KQML DAMLndash service models like DAML-Sndash social convention protocols

23418 21

Working together

bull Benevolent Agentsndash assume agents are benevolent our best

interest is their best interest

bull Self-Interested Agentsndash Agents will be assumed to act to further their

own interests possibly at expense of othersndash Potential for conflict

23418 22

Example mechanism Contract Net Protocol (CNP)

bull Negotiation as a collaboration mechanism

bull Negotiation on how tasks should be shared

ndash A task (plan) may be decomposed in a hierarchy of subtasks (hierarchical planning)

ndash An agent may subcontract another agent to perform a (sub)task

Contract

Bid

agent agent

CNP

Task announceme

nt

23418 24

CNP (Contrsquod)

Contractor

Potential candidate agents

Task announcement (broadcast)

Contractor

Candidate Candidate

Bid

Bid

Phase 1 Task Announcement

- The contractor agent publicly announces a task

- Potential candidates evaluate the task according to their won skills and availability

Phase 2 Submission of Bids Proposals

- Agents that satisfy the requiremenst ie are able to perform the task send their bid proposal to the contractor

23418 25

CNP (Contrsquod)

Contractor

Selected candidate

Contractor

Contracted agent

Contract

Phase 3 Selection

- The selection of the best candidate is made by the contractor based on received bids and on the CVs of the candidates

Phase 4 Contract awarding

- A contract is established between the contractor and the selected candidate

- A privileged bilateral communication channel is established between the two agents

23418 26

Attributes of Multi-agent Systems

Apply MAS when some of the following features show up in a problem

bull Decentralizationbull Complex components often best described at

the knowledge levelbull Adaptive behaviorbull Complex interactionsbull Coordinationbull Emergent aggregate behaviors

23418 27

Applications of MAS

Advanced Manufacturing Management Systemsminus Agents as representatives of machines users business

processes etc

Intelligent Information Search on Internetminus Some agents may show learning capabilities (learn the

preferences of their users )

Intelligent security enforcement on Internetminus Agents are representative of sensors or IDSs

Shopping Agents in Electronic Commerceminus With search price comparison and bargaining capabilities

23418 28

Applications of MAS (Contrsquod)

Multi-agent auction in E-commerce

Distributed Surveillanceminus For information search or to look for special events informing

their users of relevant news

Distributed Signal Processingminus For problem diagnosis situation assessment etc in the

network

Distributed Problem Solvingminus Collaborative design scheduling and planning

23418 29

How to simulate SI for search

Example1 Ant --gt Ant Colony Optimization (ACO)

Example2 Bird Flocking --gt Particle Swarm Optimization (PSO)

23418 30

Part Ant Colony Optimization Ⅱ(ACO)

First proposed by M Dorigo 1992

23418 31

Natural Antsbull Individual ants are simple insects with

limited memory and capable of performing simple actions

bull However an ant colony expresses a complex collective behavior providing intelligent solutions to problems such asndash carrying large itemsndash forming bridgesndash finding the shortest routes from the nest

to a food source prioritizing food sources based on their distance and ease of access

23418 32

Natural Antsbull Moreover in a colony each ant has its prescribed task but

the ants can switch tasks if the collective needs it ndash Outside the nest ants can have 4 different tasks

bull Foraging searching for and retrieving food

bull Patrolling looking for food supply

bull Midden work Sorting the colony refuse pile

bull Nest maintenance work construction and clearing of chambers

ndash An antrsquos decision whether to perform a task depends onbull The Phisical State of the environment

ndash If part of the nest is damaged more ants do nest maintenance work to repair it

bull Social Interactions with other ants

Communication (direct or indirect) is necessary

23418 33

bull They establish indirect communication system based on the deposition of pheromone over the path they followndash An isolated ant moves at random but when it finds a pheromone

trail there is a high probability that this ant will decide to follow the trail

ndash An ant foraging for food deposits pheromone over its route When it finds a food source it returns to the nest reinforcing its trail

ndash So other ants have greater probability to start following this trail and thereby laying more pheromone on it

ndash This process works as a positive feedback loop system because the higher the intensity of the pheromone over a trail the higher the probability of an ant start traveling through it

How can the natural ants find the shortest path

23418 34

bull Since the route B is shorter the ants on this path will complete the travel more times and thereby lay more pheromone over it

bull The pheromone concentration on trail B will increase at a higher rate than on A and soon the ants on route A will choose to follow route B

bull Since most ants will no longer travel on route A and since the pheromone is volatile trail A will start evaporating

bull Only the shortest route will remain

23418 35

Problems of ASbull Ant System tends to converge quickly

ndash This means that its exploitation of the best solution found is too high it should be exploring solution space more

ndash Pheromone evaporationupdate rule (better rule may exist)

bull Led to extensions of the ant systemndash Elitist Strategy for Ant Systems (EAS)ndash Rank based Ant Systems (AKRANK)ndash MAX-MIN Ant system (MMAS)ndash Ant Colony System (ACS)

23418 36

Part Ⅲ Particle Swarm Optimization (PSO)

1048713Firstly Proposed by Kennedy and Eberhart 1995

ldquoOnce again nature has provided us with a technique for processing information that is at once elegant and versatilerdquo

23418 37

bull In PSO a ldquoswarmrdquo is defined as an apparently disorganized collection (population) of moving individuals that tend to cluster together while each individual seems to be moving in a random direction

Bird flocking is one of the best example of PSO in nature

23418 38

Modeling bird flocking

bull The synchrony of flocking behavior is thought to be a function of birdrsquos efforts to maintain an optimal distance between themselves and their neighborsndash Birds and fish adjust their physical movement to avoid

predators seek for food and mates ndash Humans tend to adjust our beliefs and attitudes to

conform with those of our social peers Humans change in abstract multidimensional space collision-free

23418 39

Modeling bird flocking (Contrsquod)

bull Definitions ndash Flock is a group of objects that exhibit the general

class of polarized (aligned) non-colliding aggregate motion

ndash Boid is a simulated bird-like object ie it exhibits this type of behavior It can be a fish bee dinosaur etc

bull Rules for flockingndash Cohesion Each boid fly towards the centroid of its

local flock mates (that is boid in its local neighborhood)ndash Separation Each boid keep a certain distance away

from local flock mates to avoid collisionsndash Alignment Each boid align its velocity vector and keep

velocity magnitude similar with that of the local flock

23418 40

bull Imagine a birdrsquos flock in an area where there is a single food source

bull A bird donrsquot know where the food is but it knows its distance to the food

bull The best strategy is to follow the bird that is closer to the food

bull Particles save and communicate the best solution they have found

From Bird to Particle

23418 41

Features of PSO

bull Population initialized by assigning random positions and velocities potential solutions are then flown through hyperspace

bull Each particle keeps track of its ldquobestrdquo (highest fitness) position in hyperspacendash This is called ldquopBestrdquo for an individual particlendash It is called ldquogBestrdquo for the best in the populationndash It is called ldquolBestrdquo for the best in a defined neighborhood

bull At each time step each particle stochastically accelerates toward its pBest and gBest (or lBest)

23418 42

Particle Swarm Optimization Process

Step1 Initialize population in hyperspace

Step2 Evaluate fitness of individual particles

Step3 Modify velocities based on previous best and global (or neighborhood) best

Step4 Terminate on some condition

Step5 Go to step 2

23418 43

How do particles fly

bull Combination of gBest and the pBest (lBest)ndash need a compromise

bull lBest can bendash Social the particles around are always the same no matter

where they are in spacendash Geographical the particles around are those whose distance is

the shortest

bull Global PSO vs Local PSOndash the global version converges quickly to a solution but it gets

more easily stuck in local minima

23418 44

Illustrating the velocity update schema of global PSO

23418 45

PSO Related issues

bull Controlling velocities (determining the best value for Vmax)ndash Usually set maximum velocity to dynamic range of variable

bull Usually set c1 and c2 to 2

bull Inertia weight influences tradeoff between global and local explorationndash Good approach is to reduce inertia weight during run (ie from

09 to 04 over 1000 generations)

bull Swarm Size and Neighborhood Size

23418 46

Advantages of PSO

bull Adaptation operates on velocitiesndash Most other methods operate on positionsndash Effect PSO has a builtin momentumndash Particles tend to hurdle past optima ndash an advantage

since the best positions are remembered anyway

bull Simple in conceptbull Easy to implementbull Computationally efficientbull Effective on a variety of problems

23418 47

Summarybull Swarm Intelligence (SI)

ndash an artificial intelligence technique based around the study of collective behavior in decentralized self-organized systems

bull Multi-Agent Systems (MAS)ndash A system that consists of a number of agents

which interact with one-anotherndash Communication Coordination

Collaboration

23418 48

bull Ant Colony Optimization (ACO)ndash Inspired by ant colony foragingndash pheromone as heuristic information

(stigmergy)ndash Iteration between ConstructAntSolutions and

UpdatePheromones

bull Particle Swarm Optimization (PSO)ndash Inspired by bird flockingndash Heuristic information results from partnersndash Particle Velocity Update

• Collective Intelligence
• Outline
• The Computational Beauty of Nature
• Examples of Collective Behavior in Nature and Society
• Emergence
• Biological motivation Insect Societies
• Insect Societies
• Self Organization
• Stigmergy
• Flocks Herds and Schools
• Boid rules
• Slide 12
• Swarm Intelligence (Contrsquod)
• Components of SI
• Characteristics of SI
• The Web becomes a Giant Brain Some see the Web evolving into a collective brain for humankind
• What is Multi-agent Systems
• What is MAS(Contrsquod)
• Slide 19
• Communication models
• Working together
• Example mechanism Contract Net Protocol (CNP)
• Slide 23
• CNP (Contrsquod)
• Slide 25
• Attributes of Multi-agent Systems
• Applications of MAS
• Applications of MAS (Contrsquod)
• How to simulate SI for search
• Part Ⅱ Ant Colony Optimization (ACO)
• Natural Ants
• Slide 32
• How can the natural ants find the shortest path
• Slide 34
• Problems of AS
• Part Ⅲ Particle Swarm Optimization (PSO)
• Slide 37
• Modeling bird flocking
• Modeling bird flocking (Contrsquod)
• From Bird to Particle
• Features of PSO
• Particle Swarm Optimization Process
• How do particles fly
• Slide 44
• PSO Related issues
• Advantages of PSO
• Summary
• Slide 48

23418 16

The Web becomes a Giant Brain

Some see the Web evolving

intoa collective

brain for humankind

23418 17

What is Multi-agent Systemsbull A set of agents which interact in a common environmentbull Focus on the collaborative resolution of global

problems by a set of distributed entitiesbull Agent attempt to satisfy their own local goals as well as

the collaborative global goals bull To successfully interact they will require the ability to

cooperate coordinate and negotiate with each other much as people do

23418 18

What is MAS(Contrsquod)

bull MAS as seen from Distributed AIndash a loosely coupled network of entities that work

together to find answers to problems that are beyond the individual capabilities or knowledge of each entity

bull A more general meaningndash systems composed of autonomous components

that exhibit the following characteristicsbull each agent has incomplete capabilities to solve a problembull there is no global system controlbull data is decentralizedbull computation is asynchronous

23418 19

bull Traditionalndash Client-serverndash Low-level

messagesndash Synchronousndash Can not do the job

bull Agent breakthroughsndash Peer-to-peer

topologyndash Blackboard

coordination modelndash Encapsulated

messagingndash High-level message

protocols

Client ServerFunction(Parameters)

Return(Parameters)

Traditional Software

IntelligentAgents

IntelligentAgents

IntelligentAgents

Blackboard

MessageReply

Agents IntelligentAgentsIntelligent

Agents

IntelligentAgentsIntelligent

Agents

IntelligentAgentsIntelligent

Agents

23418 20

Communication models

bull Theoretical models Speech act theory

bull Practical modelsndash shared languages like KIF KQML DAMLndash service models like DAML-Sndash social convention protocols

23418 21

Working together

bull Benevolent Agentsndash assume agents are benevolent our best

interest is their best interest

bull Self-Interested Agentsndash Agents will be assumed to act to further their

own interests possibly at expense of othersndash Potential for conflict

23418 22

Example mechanism Contract Net Protocol (CNP)

bull Negotiation as a collaboration mechanism

bull Negotiation on how tasks should be shared

ndash A task (plan) may be decomposed in a hierarchy of subtasks (hierarchical planning)

ndash An agent may subcontract another agent to perform a (sub)task

Contract

Bid

agent agent

CNP

Task announceme

nt

23418 24

CNP (Contrsquod)

Contractor

Potential candidate agents

Task announcement (broadcast)

Contractor

Candidate Candidate

Bid

Bid

Phase 1 Task Announcement

- The contractor agent publicly announces a task

- Potential candidates evaluate the task according to their won skills and availability

Phase 2 Submission of Bids Proposals

- Agents that satisfy the requiremenst ie are able to perform the task send their bid proposal to the contractor

23418 25

CNP (Contrsquod)

Contractor

Selected candidate

Contractor

Contracted agent

Contract

Phase 3 Selection

- The selection of the best candidate is made by the contractor based on received bids and on the CVs of the candidates

Phase 4 Contract awarding

- A contract is established between the contractor and the selected candidate

- A privileged bilateral communication channel is established between the two agents

23418 26

Attributes of Multi-agent Systems

Apply MAS when some of the following features show up in a problem

bull Decentralizationbull Complex components often best described at

the knowledge levelbull Adaptive behaviorbull Complex interactionsbull Coordinationbull Emergent aggregate behaviors

23418 27

Applications of MAS

Advanced Manufacturing Management Systemsminus Agents as representatives of machines users business

processes etc

Intelligent Information Search on Internetminus Some agents may show learning capabilities (learn the

preferences of their users )

Intelligent security enforcement on Internetminus Agents are representative of sensors or IDSs

Shopping Agents in Electronic Commerceminus With search price comparison and bargaining capabilities

23418 28

Applications of MAS (Contrsquod)

Multi-agent auction in E-commerce

Distributed Surveillanceminus For information search or to look for special events informing

their users of relevant news

Distributed Signal Processingminus For problem diagnosis situation assessment etc in the

network

Distributed Problem Solvingminus Collaborative design scheduling and planning

23418 29

How to simulate SI for search

Example1 Ant --gt Ant Colony Optimization (ACO)

Example2 Bird Flocking --gt Particle Swarm Optimization (PSO)

23418 30

Part Ant Colony Optimization Ⅱ(ACO)

First proposed by M Dorigo 1992

23418 31

Natural Antsbull Individual ants are simple insects with

limited memory and capable of performing simple actions

bull However an ant colony expresses a complex collective behavior providing intelligent solutions to problems such asndash carrying large itemsndash forming bridgesndash finding the shortest routes from the nest

to a food source prioritizing food sources based on their distance and ease of access

23418 32

Natural Antsbull Moreover in a colony each ant has its prescribed task but

the ants can switch tasks if the collective needs it ndash Outside the nest ants can have 4 different tasks

bull Foraging searching for and retrieving food

bull Patrolling looking for food supply

bull Midden work Sorting the colony refuse pile

bull Nest maintenance work construction and clearing of chambers

ndash An antrsquos decision whether to perform a task depends onbull The Phisical State of the environment

ndash If part of the nest is damaged more ants do nest maintenance work to repair it

bull Social Interactions with other ants

Communication (direct or indirect) is necessary

23418 33

bull They establish indirect communication system based on the deposition of pheromone over the path they followndash An isolated ant moves at random but when it finds a pheromone

trail there is a high probability that this ant will decide to follow the trail

ndash An ant foraging for food deposits pheromone over its route When it finds a food source it returns to the nest reinforcing its trail

ndash So other ants have greater probability to start following this trail and thereby laying more pheromone on it

ndash This process works as a positive feedback loop system because the higher the intensity of the pheromone over a trail the higher the probability of an ant start traveling through it

How can the natural ants find the shortest path

23418 34

bull Since the route B is shorter the ants on this path will complete the travel more times and thereby lay more pheromone over it

bull The pheromone concentration on trail B will increase at a higher rate than on A and soon the ants on route A will choose to follow route B

bull Since most ants will no longer travel on route A and since the pheromone is volatile trail A will start evaporating

bull Only the shortest route will remain

23418 35

Problems of ASbull Ant System tends to converge quickly

ndash This means that its exploitation of the best solution found is too high it should be exploring solution space more

ndash Pheromone evaporationupdate rule (better rule may exist)

bull Led to extensions of the ant systemndash Elitist Strategy for Ant Systems (EAS)ndash Rank based Ant Systems (AKRANK)ndash MAX-MIN Ant system (MMAS)ndash Ant Colony System (ACS)

23418 36

Part Ⅲ Particle Swarm Optimization (PSO)

1048713Firstly Proposed by Kennedy and Eberhart 1995

ldquoOnce again nature has provided us with a technique for processing information that is at once elegant and versatilerdquo

23418 37

bull In PSO a ldquoswarmrdquo is defined as an apparently disorganized collection (population) of moving individuals that tend to cluster together while each individual seems to be moving in a random direction

Bird flocking is one of the best example of PSO in nature

23418 38

Modeling bird flocking

bull The synchrony of flocking behavior is thought to be a function of birdrsquos efforts to maintain an optimal distance between themselves and their neighborsndash Birds and fish adjust their physical movement to avoid

predators seek for food and mates ndash Humans tend to adjust our beliefs and attitudes to

conform with those of our social peers Humans change in abstract multidimensional space collision-free

23418 39

Modeling bird flocking (Contrsquod)

bull Definitions ndash Flock is a group of objects that exhibit the general

class of polarized (aligned) non-colliding aggregate motion

ndash Boid is a simulated bird-like object ie it exhibits this type of behavior It can be a fish bee dinosaur etc

bull Rules for flockingndash Cohesion Each boid fly towards the centroid of its

local flock mates (that is boid in its local neighborhood)ndash Separation Each boid keep a certain distance away

from local flock mates to avoid collisionsndash Alignment Each boid align its velocity vector and keep

velocity magnitude similar with that of the local flock

23418 40

bull Imagine a birdrsquos flock in an area where there is a single food source

bull A bird donrsquot know where the food is but it knows its distance to the food

bull The best strategy is to follow the bird that is closer to the food

bull Particles save and communicate the best solution they have found

From Bird to Particle

23418 41

Features of PSO

bull Population initialized by assigning random positions and velocities potential solutions are then flown through hyperspace

bull Each particle keeps track of its ldquobestrdquo (highest fitness) position in hyperspacendash This is called ldquopBestrdquo for an individual particlendash It is called ldquogBestrdquo for the best in the populationndash It is called ldquolBestrdquo for the best in a defined neighborhood

bull At each time step each particle stochastically accelerates toward its pBest and gBest (or lBest)

23418 42

Particle Swarm Optimization Process

Step1 Initialize population in hyperspace

Step2 Evaluate fitness of individual particles

Step3 Modify velocities based on previous best and global (or neighborhood) best

Step4 Terminate on some condition

Step5 Go to step 2

23418 43

How do particles fly

bull Combination of gBest and the pBest (lBest)ndash need a compromise

bull lBest can bendash Social the particles around are always the same no matter

where they are in spacendash Geographical the particles around are those whose distance is

the shortest

bull Global PSO vs Local PSOndash the global version converges quickly to a solution but it gets

more easily stuck in local minima

23418 44

Illustrating the velocity update schema of global PSO

23418 45

PSO Related issues

bull Controlling velocities (determining the best value for Vmax)ndash Usually set maximum velocity to dynamic range of variable

bull Usually set c1 and c2 to 2

bull Inertia weight influences tradeoff between global and local explorationndash Good approach is to reduce inertia weight during run (ie from

09 to 04 over 1000 generations)

bull Swarm Size and Neighborhood Size

23418 46

Advantages of PSO

bull Adaptation operates on velocitiesndash Most other methods operate on positionsndash Effect PSO has a builtin momentumndash Particles tend to hurdle past optima ndash an advantage

since the best positions are remembered anyway

bull Simple in conceptbull Easy to implementbull Computationally efficientbull Effective on a variety of problems

23418 47

Summarybull Swarm Intelligence (SI)

ndash an artificial intelligence technique based around the study of collective behavior in decentralized self-organized systems

bull Multi-Agent Systems (MAS)ndash A system that consists of a number of agents

which interact with one-anotherndash Communication Coordination

Collaboration

23418 48

bull Ant Colony Optimization (ACO)ndash Inspired by ant colony foragingndash pheromone as heuristic information

(stigmergy)ndash Iteration between ConstructAntSolutions and

UpdatePheromones

bull Particle Swarm Optimization (PSO)ndash Inspired by bird flockingndash Heuristic information results from partnersndash Particle Velocity Update

• Collective Intelligence
• Outline
• The Computational Beauty of Nature
• Examples of Collective Behavior in Nature and Society
• Emergence
• Biological motivation Insect Societies
• Insect Societies
• Self Organization
• Stigmergy
• Flocks Herds and Schools
• Boid rules
• Slide 12
• Swarm Intelligence (Contrsquod)
• Components of SI
• Characteristics of SI
• The Web becomes a Giant Brain Some see the Web evolving into a collective brain for humankind
• What is Multi-agent Systems
• What is MAS(Contrsquod)
• Slide 19
• Communication models
• Working together
• Example mechanism Contract Net Protocol (CNP)
• Slide 23
• CNP (Contrsquod)
• Slide 25
• Attributes of Multi-agent Systems
• Applications of MAS
• Applications of MAS (Contrsquod)
• How to simulate SI for search
• Part Ⅱ Ant Colony Optimization (ACO)
• Natural Ants
• Slide 32
• How can the natural ants find the shortest path
• Slide 34
• Problems of AS
• Part Ⅲ Particle Swarm Optimization (PSO)
• Slide 37
• Modeling bird flocking
• Modeling bird flocking (Contrsquod)
• From Bird to Particle
• Features of PSO
• Particle Swarm Optimization Process
• How do particles fly
• Slide 44
• PSO Related issues
• Advantages of PSO
• Summary
• Slide 48

23418 17

What is Multi-agent Systemsbull A set of agents which interact in a common environmentbull Focus on the collaborative resolution of global

problems by a set of distributed entitiesbull Agent attempt to satisfy their own local goals as well as

the collaborative global goals bull To successfully interact they will require the ability to

cooperate coordinate and negotiate with each other much as people do

23418 18

What is MAS(Contrsquod)

bull MAS as seen from Distributed AIndash a loosely coupled network of entities that work

together to find answers to problems that are beyond the individual capabilities or knowledge of each entity

bull A more general meaningndash systems composed of autonomous components

that exhibit the following characteristicsbull each agent has incomplete capabilities to solve a problembull there is no global system controlbull data is decentralizedbull computation is asynchronous

23418 19

bull Traditionalndash Client-serverndash Low-level

messagesndash Synchronousndash Can not do the job

bull Agent breakthroughsndash Peer-to-peer

topologyndash Blackboard

coordination modelndash Encapsulated

messagingndash High-level message

protocols

Client ServerFunction(Parameters)

Return(Parameters)

Traditional Software

IntelligentAgents

IntelligentAgents

IntelligentAgents

Blackboard

MessageReply

Agents IntelligentAgentsIntelligent

Agents

IntelligentAgentsIntelligent

Agents

IntelligentAgentsIntelligent

Agents

23418 20

Communication models

bull Theoretical models Speech act theory

bull Practical modelsndash shared languages like KIF KQML DAMLndash service models like DAML-Sndash social convention protocols

23418 21

Working together

bull Benevolent Agentsndash assume agents are benevolent our best

interest is their best interest

bull Self-Interested Agentsndash Agents will be assumed to act to further their

own interests possibly at expense of othersndash Potential for conflict

23418 22

Example mechanism Contract Net Protocol (CNP)

bull Negotiation as a collaboration mechanism

bull Negotiation on how tasks should be shared

ndash A task (plan) may be decomposed in a hierarchy of subtasks (hierarchical planning)

ndash An agent may subcontract another agent to perform a (sub)task

Contract

Bid

agent agent

CNP

Task announceme

nt

23418 24

CNP (Contrsquod)

Contractor

Potential candidate agents

Task announcement (broadcast)

Contractor

Candidate Candidate

Bid

Bid

Phase 1 Task Announcement

- The contractor agent publicly announces a task

- Potential candidates evaluate the task according to their won skills and availability

Phase 2 Submission of Bids Proposals

- Agents that satisfy the requiremenst ie are able to perform the task send their bid proposal to the contractor

23418 25

CNP (Contrsquod)

Contractor

Selected candidate

Contractor

Contracted agent

Contract

Phase 3 Selection

- The selection of the best candidate is made by the contractor based on received bids and on the CVs of the candidates

Phase 4 Contract awarding

- A contract is established between the contractor and the selected candidate

- A privileged bilateral communication channel is established between the two agents

23418 26

Attributes of Multi-agent Systems

Apply MAS when some of the following features show up in a problem

bull Decentralizationbull Complex components often best described at

the knowledge levelbull Adaptive behaviorbull Complex interactionsbull Coordinationbull Emergent aggregate behaviors

23418 27

Applications of MAS

Advanced Manufacturing Management Systemsminus Agents as representatives of machines users business

processes etc

Intelligent Information Search on Internetminus Some agents may show learning capabilities (learn the

preferences of their users )

Intelligent security enforcement on Internetminus Agents are representative of sensors or IDSs

Shopping Agents in Electronic Commerceminus With search price comparison and bargaining capabilities

23418 28

Applications of MAS (Contrsquod)

Multi-agent auction in E-commerce

Distributed Surveillanceminus For information search or to look for special events informing

their users of relevant news

Distributed Signal Processingminus For problem diagnosis situation assessment etc in the

network

Distributed Problem Solvingminus Collaborative design scheduling and planning

23418 29

How to simulate SI for search

Example1 Ant --gt Ant Colony Optimization (ACO)

Example2 Bird Flocking --gt Particle Swarm Optimization (PSO)

23418 30

Part Ant Colony Optimization Ⅱ(ACO)

First proposed by M Dorigo 1992

23418 31

Natural Antsbull Individual ants are simple insects with

limited memory and capable of performing simple actions

bull However an ant colony expresses a complex collective behavior providing intelligent solutions to problems such asndash carrying large itemsndash forming bridgesndash finding the shortest routes from the nest

to a food source prioritizing food sources based on their distance and ease of access

23418 32

Natural Antsbull Moreover in a colony each ant has its prescribed task but

the ants can switch tasks if the collective needs it ndash Outside the nest ants can have 4 different tasks

bull Foraging searching for and retrieving food

bull Patrolling looking for food supply

bull Midden work Sorting the colony refuse pile

bull Nest maintenance work construction and clearing of chambers

ndash An antrsquos decision whether to perform a task depends onbull The Phisical State of the environment

ndash If part of the nest is damaged more ants do nest maintenance work to repair it

bull Social Interactions with other ants

Communication (direct or indirect) is necessary

23418 33

bull They establish indirect communication system based on the deposition of pheromone over the path they followndash An isolated ant moves at random but when it finds a pheromone

trail there is a high probability that this ant will decide to follow the trail

ndash An ant foraging for food deposits pheromone over its route When it finds a food source it returns to the nest reinforcing its trail

ndash So other ants have greater probability to start following this trail and thereby laying more pheromone on it

ndash This process works as a positive feedback loop system because the higher the intensity of the pheromone over a trail the higher the probability of an ant start traveling through it

How can the natural ants find the shortest path

23418 34

bull Since the route B is shorter the ants on this path will complete the travel more times and thereby lay more pheromone over it

bull The pheromone concentration on trail B will increase at a higher rate than on A and soon the ants on route A will choose to follow route B

bull Since most ants will no longer travel on route A and since the pheromone is volatile trail A will start evaporating

bull Only the shortest route will remain

23418 35

Problems of ASbull Ant System tends to converge quickly

ndash This means that its exploitation of the best solution found is too high it should be exploring solution space more

ndash Pheromone evaporationupdate rule (better rule may exist)

bull Led to extensions of the ant systemndash Elitist Strategy for Ant Systems (EAS)ndash Rank based Ant Systems (AKRANK)ndash MAX-MIN Ant system (MMAS)ndash Ant Colony System (ACS)

23418 36

Part Ⅲ Particle Swarm Optimization (PSO)

1048713Firstly Proposed by Kennedy and Eberhart 1995

ldquoOnce again nature has provided us with a technique for processing information that is at once elegant and versatilerdquo

23418 37

bull In PSO a ldquoswarmrdquo is defined as an apparently disorganized collection (population) of moving individuals that tend to cluster together while each individual seems to be moving in a random direction

Bird flocking is one of the best example of PSO in nature

23418 38

Modeling bird flocking

bull The synchrony of flocking behavior is thought to be a function of birdrsquos efforts to maintain an optimal distance between themselves and their neighborsndash Birds and fish adjust their physical movement to avoid

predators seek for food and mates ndash Humans tend to adjust our beliefs and attitudes to

conform with those of our social peers Humans change in abstract multidimensional space collision-free

23418 39

Modeling bird flocking (Contrsquod)

bull Definitions ndash Flock is a group of objects that exhibit the general

class of polarized (aligned) non-colliding aggregate motion

ndash Boid is a simulated bird-like object ie it exhibits this type of behavior It can be a fish bee dinosaur etc

bull Rules for flockingndash Cohesion Each boid fly towards the centroid of its

local flock mates (that is boid in its local neighborhood)ndash Separation Each boid keep a certain distance away

from local flock mates to avoid collisionsndash Alignment Each boid align its velocity vector and keep

velocity magnitude similar with that of the local flock

23418 40

bull Imagine a birdrsquos flock in an area where there is a single food source

bull A bird donrsquot know where the food is but it knows its distance to the food

bull The best strategy is to follow the bird that is closer to the food

bull Particles save and communicate the best solution they have found

From Bird to Particle

23418 41

Features of PSO

bull Population initialized by assigning random positions and velocities potential solutions are then flown through hyperspace

bull Each particle keeps track of its ldquobestrdquo (highest fitness) position in hyperspacendash This is called ldquopBestrdquo for an individual particlendash It is called ldquogBestrdquo for the best in the populationndash It is called ldquolBestrdquo for the best in a defined neighborhood

bull At each time step each particle stochastically accelerates toward its pBest and gBest (or lBest)

23418 42

Particle Swarm Optimization Process

Step1 Initialize population in hyperspace

Step2 Evaluate fitness of individual particles

Step3 Modify velocities based on previous best and global (or neighborhood) best

Step4 Terminate on some condition

Step5 Go to step 2

23418 43

How do particles fly

bull Combination of gBest and the pBest (lBest)ndash need a compromise

bull lBest can bendash Social the particles around are always the same no matter

where they are in spacendash Geographical the particles around are those whose distance is

the shortest

bull Global PSO vs Local PSOndash the global version converges quickly to a solution but it gets

more easily stuck in local minima

23418 44

Illustrating the velocity update schema of global PSO

23418 45

PSO Related issues

bull Controlling velocities (determining the best value for Vmax)ndash Usually set maximum velocity to dynamic range of variable

bull Usually set c1 and c2 to 2

bull Inertia weight influences tradeoff between global and local explorationndash Good approach is to reduce inertia weight during run (ie from

09 to 04 over 1000 generations)

bull Swarm Size and Neighborhood Size

23418 46

Advantages of PSO

bull Adaptation operates on velocitiesndash Most other methods operate on positionsndash Effect PSO has a builtin momentumndash Particles tend to hurdle past optima ndash an advantage

since the best positions are remembered anyway

bull Simple in conceptbull Easy to implementbull Computationally efficientbull Effective on a variety of problems

23418 47

Summarybull Swarm Intelligence (SI)

ndash an artificial intelligence technique based around the study of collective behavior in decentralized self-organized systems

bull Multi-Agent Systems (MAS)ndash A system that consists of a number of agents

which interact with one-anotherndash Communication Coordination

Collaboration

23418 48

bull Ant Colony Optimization (ACO)ndash Inspired by ant colony foragingndash pheromone as heuristic information

(stigmergy)ndash Iteration between ConstructAntSolutions and

UpdatePheromones

bull Particle Swarm Optimization (PSO)ndash Inspired by bird flockingndash Heuristic information results from partnersndash Particle Velocity Update

• Collective Intelligence
• Outline
• The Computational Beauty of Nature
• Examples of Collective Behavior in Nature and Society
• Emergence
• Biological motivation Insect Societies
• Insect Societies
• Self Organization
• Stigmergy
• Flocks Herds and Schools
• Boid rules
• Slide 12
• Swarm Intelligence (Contrsquod)
• Components of SI
• Characteristics of SI
• The Web becomes a Giant Brain Some see the Web evolving into a collective brain for humankind
• What is Multi-agent Systems
• What is MAS(Contrsquod)
• Slide 19
• Communication models
• Working together
• Example mechanism Contract Net Protocol (CNP)
• Slide 23
• CNP (Contrsquod)
• Slide 25
• Attributes of Multi-agent Systems
• Applications of MAS
• Applications of MAS (Contrsquod)
• How to simulate SI for search
• Part Ⅱ Ant Colony Optimization (ACO)
• Natural Ants
• Slide 32
• How can the natural ants find the shortest path
• Slide 34
• Problems of AS
• Part Ⅲ Particle Swarm Optimization (PSO)
• Slide 37
• Modeling bird flocking
• Modeling bird flocking (Contrsquod)
• From Bird to Particle
• Features of PSO
• Particle Swarm Optimization Process
• How do particles fly
• Slide 44
• PSO Related issues
• Advantages of PSO
• Summary
• Slide 48

23418 18

What is MAS(Contrsquod)

bull MAS as seen from Distributed AIndash a loosely coupled network of entities that work

together to find answers to problems that are beyond the individual capabilities or knowledge of each entity

bull A more general meaningndash systems composed of autonomous components

that exhibit the following characteristicsbull each agent has incomplete capabilities to solve a problembull there is no global system controlbull data is decentralizedbull computation is asynchronous

23418 19

bull Traditionalndash Client-serverndash Low-level

messagesndash Synchronousndash Can not do the job

bull Agent breakthroughsndash Peer-to-peer

topologyndash Blackboard

coordination modelndash Encapsulated

messagingndash High-level message

protocols

Client ServerFunction(Parameters)

Return(Parameters)

Traditional Software

IntelligentAgents

IntelligentAgents

IntelligentAgents

Blackboard

MessageReply

Agents IntelligentAgentsIntelligent

Agents

IntelligentAgentsIntelligent

Agents

IntelligentAgentsIntelligent

Agents

23418 20

Communication models

bull Theoretical models Speech act theory

bull Practical modelsndash shared languages like KIF KQML DAMLndash service models like DAML-Sndash social convention protocols

23418 21

Working together

bull Benevolent Agentsndash assume agents are benevolent our best

interest is their best interest

bull Self-Interested Agentsndash Agents will be assumed to act to further their

own interests possibly at expense of othersndash Potential for conflict

23418 22

Example mechanism Contract Net Protocol (CNP)

bull Negotiation as a collaboration mechanism

bull Negotiation on how tasks should be shared

ndash A task (plan) may be decomposed in a hierarchy of subtasks (hierarchical planning)

ndash An agent may subcontract another agent to perform a (sub)task

Contract

Bid

agent agent

CNP

Task announceme

nt

23418 24

CNP (Contrsquod)

Contractor

Potential candidate agents

Task announcement (broadcast)

Contractor

Candidate Candidate

Bid

Bid

Phase 1 Task Announcement

- The contractor agent publicly announces a task

- Potential candidates evaluate the task according to their won skills and availability

Phase 2 Submission of Bids Proposals

- Agents that satisfy the requiremenst ie are able to perform the task send their bid proposal to the contractor

23418 25

CNP (Contrsquod)

Contractor

Selected candidate

Contractor

Contracted agent

Contract

Phase 3 Selection

- The selection of the best candidate is made by the contractor based on received bids and on the CVs of the candidates

Phase 4 Contract awarding

- A contract is established between the contractor and the selected candidate

- A privileged bilateral communication channel is established between the two agents

23418 26

Attributes of Multi-agent Systems

Apply MAS when some of the following features show up in a problem

bull Decentralizationbull Complex components often best described at

the knowledge levelbull Adaptive behaviorbull Complex interactionsbull Coordinationbull Emergent aggregate behaviors

23418 27

Applications of MAS

Advanced Manufacturing Management Systemsminus Agents as representatives of machines users business

processes etc

Intelligent Information Search on Internetminus Some agents may show learning capabilities (learn the

preferences of their users )

Intelligent security enforcement on Internetminus Agents are representative of sensors or IDSs

Shopping Agents in Electronic Commerceminus With search price comparison and bargaining capabilities

23418 28

Applications of MAS (Contrsquod)

Multi-agent auction in E-commerce

Distributed Surveillanceminus For information search or to look for special events informing

their users of relevant news

Distributed Signal Processingminus For problem diagnosis situation assessment etc in the

network

Distributed Problem Solvingminus Collaborative design scheduling and planning

23418 29

How to simulate SI for search

Example1 Ant --gt Ant Colony Optimization (ACO)

Example2 Bird Flocking --gt Particle Swarm Optimization (PSO)

23418 30

Part Ant Colony Optimization Ⅱ(ACO)

First proposed by M Dorigo 1992

23418 31

Natural Antsbull Individual ants are simple insects with

limited memory and capable of performing simple actions

bull However an ant colony expresses a complex collective behavior providing intelligent solutions to problems such asndash carrying large itemsndash forming bridgesndash finding the shortest routes from the nest

to a food source prioritizing food sources based on their distance and ease of access

23418 32

Natural Antsbull Moreover in a colony each ant has its prescribed task but

the ants can switch tasks if the collective needs it ndash Outside the nest ants can have 4 different tasks

bull Foraging searching for and retrieving food

bull Patrolling looking for food supply

bull Midden work Sorting the colony refuse pile

bull Nest maintenance work construction and clearing of chambers

ndash An antrsquos decision whether to perform a task depends onbull The Phisical State of the environment

ndash If part of the nest is damaged more ants do nest maintenance work to repair it

bull Social Interactions with other ants

Communication (direct or indirect) is necessary

23418 33

bull They establish indirect communication system based on the deposition of pheromone over the path they followndash An isolated ant moves at random but when it finds a pheromone

trail there is a high probability that this ant will decide to follow the trail

ndash An ant foraging for food deposits pheromone over its route When it finds a food source it returns to the nest reinforcing its trail

ndash So other ants have greater probability to start following this trail and thereby laying more pheromone on it

ndash This process works as a positive feedback loop system because the higher the intensity of the pheromone over a trail the higher the probability of an ant start traveling through it

How can the natural ants find the shortest path

23418 34

bull Since the route B is shorter the ants on this path will complete the travel more times and thereby lay more pheromone over it

bull The pheromone concentration on trail B will increase at a higher rate than on A and soon the ants on route A will choose to follow route B

bull Since most ants will no longer travel on route A and since the pheromone is volatile trail A will start evaporating

bull Only the shortest route will remain

23418 35

Problems of ASbull Ant System tends to converge quickly

ndash This means that its exploitation of the best solution found is too high it should be exploring solution space more

ndash Pheromone evaporationupdate rule (better rule may exist)

bull Led to extensions of the ant systemndash Elitist Strategy for Ant Systems (EAS)ndash Rank based Ant Systems (AKRANK)ndash MAX-MIN Ant system (MMAS)ndash Ant Colony System (ACS)

23418 36

Part Ⅲ Particle Swarm Optimization (PSO)

1048713Firstly Proposed by Kennedy and Eberhart 1995

ldquoOnce again nature has provided us with a technique for processing information that is at once elegant and versatilerdquo

23418 37

bull In PSO a ldquoswarmrdquo is defined as an apparently disorganized collection (population) of moving individuals that tend to cluster together while each individual seems to be moving in a random direction

Bird flocking is one of the best example of PSO in nature

23418 38

Modeling bird flocking

bull The synchrony of flocking behavior is thought to be a function of birdrsquos efforts to maintain an optimal distance between themselves and their neighborsndash Birds and fish adjust their physical movement to avoid

predators seek for food and mates ndash Humans tend to adjust our beliefs and attitudes to

conform with those of our social peers Humans change in abstract multidimensional space collision-free

23418 39

Modeling bird flocking (Contrsquod)

bull Definitions ndash Flock is a group of objects that exhibit the general

class of polarized (aligned) non-colliding aggregate motion

ndash Boid is a simulated bird-like object ie it exhibits this type of behavior It can be a fish bee dinosaur etc

bull Rules for flockingndash Cohesion Each boid fly towards the centroid of its

local flock mates (that is boid in its local neighborhood)ndash Separation Each boid keep a certain distance away

from local flock mates to avoid collisionsndash Alignment Each boid align its velocity vector and keep

velocity magnitude similar with that of the local flock

23418 40

bull Imagine a birdrsquos flock in an area where there is a single food source

bull A bird donrsquot know where the food is but it knows its distance to the food

bull The best strategy is to follow the bird that is closer to the food

bull Particles save and communicate the best solution they have found

From Bird to Particle

23418 41

Features of PSO

bull Population initialized by assigning random positions and velocities potential solutions are then flown through hyperspace

bull Each particle keeps track of its ldquobestrdquo (highest fitness) position in hyperspacendash This is called ldquopBestrdquo for an individual particlendash It is called ldquogBestrdquo for the best in the populationndash It is called ldquolBestrdquo for the best in a defined neighborhood

bull At each time step each particle stochastically accelerates toward its pBest and gBest (or lBest)

23418 42

Particle Swarm Optimization Process

Step1 Initialize population in hyperspace

Step2 Evaluate fitness of individual particles

Step3 Modify velocities based on previous best and global (or neighborhood) best

Step4 Terminate on some condition

Step5 Go to step 2

23418 43

How do particles fly

bull Combination of gBest and the pBest (lBest)ndash need a compromise

bull lBest can bendash Social the particles around are always the same no matter

where they are in spacendash Geographical the particles around are those whose distance is

the shortest

bull Global PSO vs Local PSOndash the global version converges quickly to a solution but it gets

more easily stuck in local minima

23418 44

Illustrating the velocity update schema of global PSO

23418 45

PSO Related issues

bull Controlling velocities (determining the best value for Vmax)ndash Usually set maximum velocity to dynamic range of variable

bull Usually set c1 and c2 to 2

bull Inertia weight influences tradeoff between global and local explorationndash Good approach is to reduce inertia weight during run (ie from

09 to 04 over 1000 generations)

bull Swarm Size and Neighborhood Size

23418 46

Advantages of PSO

bull Adaptation operates on velocitiesndash Most other methods operate on positionsndash Effect PSO has a builtin momentumndash Particles tend to hurdle past optima ndash an advantage

since the best positions are remembered anyway

bull Simple in conceptbull Easy to implementbull Computationally efficientbull Effective on a variety of problems

23418 47

Summarybull Swarm Intelligence (SI)

ndash an artificial intelligence technique based around the study of collective behavior in decentralized self-organized systems

bull Multi-Agent Systems (MAS)ndash A system that consists of a number of agents

which interact with one-anotherndash Communication Coordination

Collaboration

23418 48

bull Ant Colony Optimization (ACO)ndash Inspired by ant colony foragingndash pheromone as heuristic information

(stigmergy)ndash Iteration between ConstructAntSolutions and

UpdatePheromones

bull Particle Swarm Optimization (PSO)ndash Inspired by bird flockingndash Heuristic information results from partnersndash Particle Velocity Update

• Collective Intelligence
• Outline
• The Computational Beauty of Nature
• Examples of Collective Behavior in Nature and Society
• Emergence
• Biological motivation Insect Societies
• Insect Societies
• Self Organization
• Stigmergy
• Flocks Herds and Schools
• Boid rules
• Slide 12
• Swarm Intelligence (Contrsquod)
• Components of SI
• Characteristics of SI
• The Web becomes a Giant Brain Some see the Web evolving into a collective brain for humankind
• What is Multi-agent Systems
• What is MAS(Contrsquod)
• Slide 19
• Communication models
• Working together
• Example mechanism Contract Net Protocol (CNP)
• Slide 23
• CNP (Contrsquod)
• Slide 25
• Attributes of Multi-agent Systems
• Applications of MAS
• Applications of MAS (Contrsquod)
• How to simulate SI for search
• Part Ⅱ Ant Colony Optimization (ACO)
• Natural Ants
• Slide 32
• How can the natural ants find the shortest path
• Slide 34
• Problems of AS
• Part Ⅲ Particle Swarm Optimization (PSO)
• Slide 37
• Modeling bird flocking
• Modeling bird flocking (Contrsquod)
• From Bird to Particle
• Features of PSO
• Particle Swarm Optimization Process
• How do particles fly
• Slide 44
• PSO Related issues
• Advantages of PSO
• Summary
• Slide 48

23418 19

bull Traditionalndash Client-serverndash Low-level

messagesndash Synchronousndash Can not do the job

bull Agent breakthroughsndash Peer-to-peer

topologyndash Blackboard

coordination modelndash Encapsulated

messagingndash High-level message

protocols

Client ServerFunction(Parameters)

Return(Parameters)

Traditional Software

IntelligentAgents

IntelligentAgents

IntelligentAgents

Blackboard

MessageReply

Agents IntelligentAgentsIntelligent

Agents

IntelligentAgentsIntelligent

Agents

IntelligentAgentsIntelligent

Agents

23418 20

Communication models

bull Theoretical models Speech act theory

bull Practical modelsndash shared languages like KIF KQML DAMLndash service models like DAML-Sndash social convention protocols

23418 21

Working together

bull Benevolent Agentsndash assume agents are benevolent our best

interest is their best interest

bull Self-Interested Agentsndash Agents will be assumed to act to further their

own interests possibly at expense of othersndash Potential for conflict

23418 22

Example mechanism Contract Net Protocol (CNP)

bull Negotiation as a collaboration mechanism

bull Negotiation on how tasks should be shared

ndash A task (plan) may be decomposed in a hierarchy of subtasks (hierarchical planning)

ndash An agent may subcontract another agent to perform a (sub)task

Contract

Bid

agent agent

CNP

Task announceme

nt

23418 24

CNP (Contrsquod)

Contractor

Potential candidate agents

Task announcement (broadcast)

Contractor

Candidate Candidate

Bid

Bid

Phase 1 Task Announcement

- The contractor agent publicly announces a task

- Potential candidates evaluate the task according to their won skills and availability

Phase 2 Submission of Bids Proposals

- Agents that satisfy the requiremenst ie are able to perform the task send their bid proposal to the contractor

23418 25

CNP (Contrsquod)

Contractor

Selected candidate

Contractor

Contracted agent

Contract

Phase 3 Selection

- The selection of the best candidate is made by the contractor based on received bids and on the CVs of the candidates

Phase 4 Contract awarding

- A contract is established between the contractor and the selected candidate

- A privileged bilateral communication channel is established between the two agents

23418 26

Attributes of Multi-agent Systems

Apply MAS when some of the following features show up in a problem

bull Decentralizationbull Complex components often best described at

the knowledge levelbull Adaptive behaviorbull Complex interactionsbull Coordinationbull Emergent aggregate behaviors

23418 27

Applications of MAS

Advanced Manufacturing Management Systemsminus Agents as representatives of machines users business

processes etc

Intelligent Information Search on Internetminus Some agents may show learning capabilities (learn the

preferences of their users )

Intelligent security enforcement on Internetminus Agents are representative of sensors or IDSs

Shopping Agents in Electronic Commerceminus With search price comparison and bargaining capabilities

23418 28

Applications of MAS (Contrsquod)

Multi-agent auction in E-commerce

Distributed Surveillanceminus For information search or to look for special events informing

their users of relevant news

Distributed Signal Processingminus For problem diagnosis situation assessment etc in the

network

Distributed Problem Solvingminus Collaborative design scheduling and planning

23418 29

How to simulate SI for search

Example1 Ant --gt Ant Colony Optimization (ACO)

Example2 Bird Flocking --gt Particle Swarm Optimization (PSO)

23418 30

Part Ant Colony Optimization Ⅱ(ACO)

First proposed by M Dorigo 1992

23418 31

Natural Antsbull Individual ants are simple insects with

limited memory and capable of performing simple actions

bull However an ant colony expresses a complex collective behavior providing intelligent solutions to problems such asndash carrying large itemsndash forming bridgesndash finding the shortest routes from the nest

to a food source prioritizing food sources based on their distance and ease of access

23418 32

Natural Antsbull Moreover in a colony each ant has its prescribed task but

the ants can switch tasks if the collective needs it ndash Outside the nest ants can have 4 different tasks

bull Foraging searching for and retrieving food

bull Patrolling looking for food supply

bull Midden work Sorting the colony refuse pile

bull Nest maintenance work construction and clearing of chambers

ndash An antrsquos decision whether to perform a task depends onbull The Phisical State of the environment

ndash If part of the nest is damaged more ants do nest maintenance work to repair it

bull Social Interactions with other ants

Communication (direct or indirect) is necessary

23418 33

bull They establish indirect communication system based on the deposition of pheromone over the path they followndash An isolated ant moves at random but when it finds a pheromone

trail there is a high probability that this ant will decide to follow the trail

ndash An ant foraging for food deposits pheromone over its route When it finds a food source it returns to the nest reinforcing its trail

ndash So other ants have greater probability to start following this trail and thereby laying more pheromone on it

ndash This process works as a positive feedback loop system because the higher the intensity of the pheromone over a trail the higher the probability of an ant start traveling through it

How can the natural ants find the shortest path

23418 34

bull Since the route B is shorter the ants on this path will complete the travel more times and thereby lay more pheromone over it

bull The pheromone concentration on trail B will increase at a higher rate than on A and soon the ants on route A will choose to follow route B

bull Since most ants will no longer travel on route A and since the pheromone is volatile trail A will start evaporating

bull Only the shortest route will remain

23418 35

Problems of ASbull Ant System tends to converge quickly

ndash This means that its exploitation of the best solution found is too high it should be exploring solution space more

ndash Pheromone evaporationupdate rule (better rule may exist)

bull Led to extensions of the ant systemndash Elitist Strategy for Ant Systems (EAS)ndash Rank based Ant Systems (AKRANK)ndash MAX-MIN Ant system (MMAS)ndash Ant Colony System (ACS)

23418 36

Part Ⅲ Particle Swarm Optimization (PSO)

1048713Firstly Proposed by Kennedy and Eberhart 1995

ldquoOnce again nature has provided us with a technique for processing information that is at once elegant and versatilerdquo

23418 37

bull In PSO a ldquoswarmrdquo is defined as an apparently disorganized collection (population) of moving individuals that tend to cluster together while each individual seems to be moving in a random direction

Bird flocking is one of the best example of PSO in nature

23418 38

Modeling bird flocking

bull The synchrony of flocking behavior is thought to be a function of birdrsquos efforts to maintain an optimal distance between themselves and their neighborsndash Birds and fish adjust their physical movement to avoid

predators seek for food and mates ndash Humans tend to adjust our beliefs and attitudes to

conform with those of our social peers Humans change in abstract multidimensional space collision-free

23418 39

Modeling bird flocking (Contrsquod)

bull Definitions ndash Flock is a group of objects that exhibit the general

class of polarized (aligned) non-colliding aggregate motion

ndash Boid is a simulated bird-like object ie it exhibits this type of behavior It can be a fish bee dinosaur etc

bull Rules for flockingndash Cohesion Each boid fly towards the centroid of its

local flock mates (that is boid in its local neighborhood)ndash Separation Each boid keep a certain distance away

from local flock mates to avoid collisionsndash Alignment Each boid align its velocity vector and keep

velocity magnitude similar with that of the local flock

23418 40

bull Imagine a birdrsquos flock in an area where there is a single food source

bull A bird donrsquot know where the food is but it knows its distance to the food

bull The best strategy is to follow the bird that is closer to the food

bull Particles save and communicate the best solution they have found

From Bird to Particle

23418 41

Features of PSO

bull Population initialized by assigning random positions and velocities potential solutions are then flown through hyperspace

bull Each particle keeps track of its ldquobestrdquo (highest fitness) position in hyperspacendash This is called ldquopBestrdquo for an individual particlendash It is called ldquogBestrdquo for the best in the populationndash It is called ldquolBestrdquo for the best in a defined neighborhood

bull At each time step each particle stochastically accelerates toward its pBest and gBest (or lBest)

23418 42

Particle Swarm Optimization Process

Step1 Initialize population in hyperspace

Step2 Evaluate fitness of individual particles

Step3 Modify velocities based on previous best and global (or neighborhood) best

Step4 Terminate on some condition

Step5 Go to step 2

23418 43

How do particles fly

bull Combination of gBest and the pBest (lBest)ndash need a compromise

bull lBest can bendash Social the particles around are always the same no matter

where they are in spacendash Geographical the particles around are those whose distance is

the shortest

bull Global PSO vs Local PSOndash the global version converges quickly to a solution but it gets

more easily stuck in local minima

23418 44

Illustrating the velocity update schema of global PSO

23418 45

PSO Related issues

bull Controlling velocities (determining the best value for Vmax)ndash Usually set maximum velocity to dynamic range of variable

bull Usually set c1 and c2 to 2

bull Inertia weight influences tradeoff between global and local explorationndash Good approach is to reduce inertia weight during run (ie from

09 to 04 over 1000 generations)

bull Swarm Size and Neighborhood Size

23418 46

Advantages of PSO

bull Adaptation operates on velocitiesndash Most other methods operate on positionsndash Effect PSO has a builtin momentumndash Particles tend to hurdle past optima ndash an advantage

since the best positions are remembered anyway

bull Simple in conceptbull Easy to implementbull Computationally efficientbull Effective on a variety of problems

23418 47

Summarybull Swarm Intelligence (SI)

ndash an artificial intelligence technique based around the study of collective behavior in decentralized self-organized systems

bull Multi-Agent Systems (MAS)ndash A system that consists of a number of agents

which interact with one-anotherndash Communication Coordination

Collaboration

23418 48

bull Ant Colony Optimization (ACO)ndash Inspired by ant colony foragingndash pheromone as heuristic information

(stigmergy)ndash Iteration between ConstructAntSolutions and

UpdatePheromones

bull Particle Swarm Optimization (PSO)ndash Inspired by bird flockingndash Heuristic information results from partnersndash Particle Velocity Update

• Collective Intelligence
• Outline
• The Computational Beauty of Nature
• Examples of Collective Behavior in Nature and Society
• Emergence
• Biological motivation Insect Societies
• Insect Societies
• Self Organization
• Stigmergy
• Flocks Herds and Schools
• Boid rules
• Slide 12
• Swarm Intelligence (Contrsquod)
• Components of SI
• Characteristics of SI
• The Web becomes a Giant Brain Some see the Web evolving into a collective brain for humankind
• What is Multi-agent Systems
• What is MAS(Contrsquod)
• Slide 19
• Communication models
• Working together
• Example mechanism Contract Net Protocol (CNP)
• Slide 23
• CNP (Contrsquod)
• Slide 25
• Attributes of Multi-agent Systems
• Applications of MAS
• Applications of MAS (Contrsquod)
• How to simulate SI for search
• Part Ⅱ Ant Colony Optimization (ACO)
• Natural Ants
• Slide 32
• How can the natural ants find the shortest path
• Slide 34
• Problems of AS
• Part Ⅲ Particle Swarm Optimization (PSO)
• Slide 37
• Modeling bird flocking
• Modeling bird flocking (Contrsquod)
• From Bird to Particle
• Features of PSO
• Particle Swarm Optimization Process
• How do particles fly
• Slide 44
• PSO Related issues
• Advantages of PSO
• Summary
• Slide 48

23418 20

Communication models

bull Theoretical models Speech act theory

bull Practical modelsndash shared languages like KIF KQML DAMLndash service models like DAML-Sndash social convention protocols

23418 21

Working together

bull Benevolent Agentsndash assume agents are benevolent our best

interest is their best interest

bull Self-Interested Agentsndash Agents will be assumed to act to further their

own interests possibly at expense of othersndash Potential for conflict

23418 22

Example mechanism Contract Net Protocol (CNP)

bull Negotiation as a collaboration mechanism

bull Negotiation on how tasks should be shared

ndash A task (plan) may be decomposed in a hierarchy of subtasks (hierarchical planning)

ndash An agent may subcontract another agent to perform a (sub)task

Contract

Bid

agent agent

CNP

Task announceme

nt

23418 24

CNP (Contrsquod)

Contractor

Potential candidate agents

Task announcement (broadcast)

Contractor

Candidate Candidate

Bid

Bid

Phase 1 Task Announcement

- The contractor agent publicly announces a task

- Potential candidates evaluate the task according to their won skills and availability

Phase 2 Submission of Bids Proposals

- Agents that satisfy the requiremenst ie are able to perform the task send their bid proposal to the contractor

23418 25

CNP (Contrsquod)

Contractor

Selected candidate

Contractor

Contracted agent

Contract

Phase 3 Selection

- The selection of the best candidate is made by the contractor based on received bids and on the CVs of the candidates

Phase 4 Contract awarding

- A contract is established between the contractor and the selected candidate

- A privileged bilateral communication channel is established between the two agents

23418 26

Attributes of Multi-agent Systems

Apply MAS when some of the following features show up in a problem

bull Decentralizationbull Complex components often best described at

the knowledge levelbull Adaptive behaviorbull Complex interactionsbull Coordinationbull Emergent aggregate behaviors

23418 27

Applications of MAS

Advanced Manufacturing Management Systemsminus Agents as representatives of machines users business

processes etc

Intelligent Information Search on Internetminus Some agents may show learning capabilities (learn the

preferences of their users )

Intelligent security enforcement on Internetminus Agents are representative of sensors or IDSs

Shopping Agents in Electronic Commerceminus With search price comparison and bargaining capabilities

23418 28

Applications of MAS (Contrsquod)

Multi-agent auction in E-commerce

Distributed Surveillanceminus For information search or to look for special events informing

their users of relevant news

Distributed Signal Processingminus For problem diagnosis situation assessment etc in the

network

Distributed Problem Solvingminus Collaborative design scheduling and planning

23418 29

How to simulate SI for search

Example1 Ant --gt Ant Colony Optimization (ACO)

Example2 Bird Flocking --gt Particle Swarm Optimization (PSO)

23418 30

Part Ant Colony Optimization Ⅱ(ACO)

First proposed by M Dorigo 1992

23418 31

Natural Antsbull Individual ants are simple insects with

limited memory and capable of performing simple actions

bull However an ant colony expresses a complex collective behavior providing intelligent solutions to problems such asndash carrying large itemsndash forming bridgesndash finding the shortest routes from the nest

to a food source prioritizing food sources based on their distance and ease of access

23418 32

Natural Antsbull Moreover in a colony each ant has its prescribed task but

the ants can switch tasks if the collective needs it ndash Outside the nest ants can have 4 different tasks

bull Foraging searching for and retrieving food

bull Patrolling looking for food supply

bull Midden work Sorting the colony refuse pile

bull Nest maintenance work construction and clearing of chambers

ndash An antrsquos decision whether to perform a task depends onbull The Phisical State of the environment

ndash If part of the nest is damaged more ants do nest maintenance work to repair it

bull Social Interactions with other ants

Communication (direct or indirect) is necessary

23418 33

bull They establish indirect communication system based on the deposition of pheromone over the path they followndash An isolated ant moves at random but when it finds a pheromone

trail there is a high probability that this ant will decide to follow the trail

ndash An ant foraging for food deposits pheromone over its route When it finds a food source it returns to the nest reinforcing its trail

ndash So other ants have greater probability to start following this trail and thereby laying more pheromone on it

ndash This process works as a positive feedback loop system because the higher the intensity of the pheromone over a trail the higher the probability of an ant start traveling through it

How can the natural ants find the shortest path

23418 34

bull Since the route B is shorter the ants on this path will complete the travel more times and thereby lay more pheromone over it

bull The pheromone concentration on trail B will increase at a higher rate than on A and soon the ants on route A will choose to follow route B

bull Since most ants will no longer travel on route A and since the pheromone is volatile trail A will start evaporating

bull Only the shortest route will remain

23418 35

Problems of ASbull Ant System tends to converge quickly

ndash This means that its exploitation of the best solution found is too high it should be exploring solution space more

ndash Pheromone evaporationupdate rule (better rule may exist)

bull Led to extensions of the ant systemndash Elitist Strategy for Ant Systems (EAS)ndash Rank based Ant Systems (AKRANK)ndash MAX-MIN Ant system (MMAS)ndash Ant Colony System (ACS)

23418 36

Part Ⅲ Particle Swarm Optimization (PSO)

1048713Firstly Proposed by Kennedy and Eberhart 1995

ldquoOnce again nature has provided us with a technique for processing information that is at once elegant and versatilerdquo

23418 37

bull In PSO a ldquoswarmrdquo is defined as an apparently disorganized collection (population) of moving individuals that tend to cluster together while each individual seems to be moving in a random direction

Bird flocking is one of the best example of PSO in nature

23418 38

Modeling bird flocking

bull The synchrony of flocking behavior is thought to be a function of birdrsquos efforts to maintain an optimal distance between themselves and their neighborsndash Birds and fish adjust their physical movement to avoid

predators seek for food and mates ndash Humans tend to adjust our beliefs and attitudes to

conform with those of our social peers Humans change in abstract multidimensional space collision-free

23418 39

Modeling bird flocking (Contrsquod)

bull Definitions ndash Flock is a group of objects that exhibit the general

class of polarized (aligned) non-colliding aggregate motion

ndash Boid is a simulated bird-like object ie it exhibits this type of behavior It can be a fish bee dinosaur etc

bull Rules for flockingndash Cohesion Each boid fly towards the centroid of its

local flock mates (that is boid in its local neighborhood)ndash Separation Each boid keep a certain distance away

from local flock mates to avoid collisionsndash Alignment Each boid align its velocity vector and keep

velocity magnitude similar with that of the local flock

23418 40

bull Imagine a birdrsquos flock in an area where there is a single food source

bull A bird donrsquot know where the food is but it knows its distance to the food

bull The best strategy is to follow the bird that is closer to the food

bull Particles save and communicate the best solution they have found

From Bird to Particle

23418 41

Features of PSO

bull Population initialized by assigning random positions and velocities potential solutions are then flown through hyperspace

bull Each particle keeps track of its ldquobestrdquo (highest fitness) position in hyperspacendash This is called ldquopBestrdquo for an individual particlendash It is called ldquogBestrdquo for the best in the populationndash It is called ldquolBestrdquo for the best in a defined neighborhood

bull At each time step each particle stochastically accelerates toward its pBest and gBest (or lBest)

23418 42

Particle Swarm Optimization Process

Step1 Initialize population in hyperspace

Step2 Evaluate fitness of individual particles

Step3 Modify velocities based on previous best and global (or neighborhood) best

Step4 Terminate on some condition

Step5 Go to step 2

23418 43

How do particles fly

bull Combination of gBest and the pBest (lBest)ndash need a compromise

bull lBest can bendash Social the particles around are always the same no matter

where they are in spacendash Geographical the particles around are those whose distance is

the shortest

bull Global PSO vs Local PSOndash the global version converges quickly to a solution but it gets

more easily stuck in local minima

23418 44

Illustrating the velocity update schema of global PSO

23418 45

PSO Related issues

bull Controlling velocities (determining the best value for Vmax)ndash Usually set maximum velocity to dynamic range of variable

bull Usually set c1 and c2 to 2

bull Inertia weight influences tradeoff between global and local explorationndash Good approach is to reduce inertia weight during run (ie from

09 to 04 over 1000 generations)

bull Swarm Size and Neighborhood Size

23418 46

Advantages of PSO

bull Adaptation operates on velocitiesndash Most other methods operate on positionsndash Effect PSO has a builtin momentumndash Particles tend to hurdle past optima ndash an advantage

since the best positions are remembered anyway

bull Simple in conceptbull Easy to implementbull Computationally efficientbull Effective on a variety of problems

23418 47

Summarybull Swarm Intelligence (SI)

ndash an artificial intelligence technique based around the study of collective behavior in decentralized self-organized systems

bull Multi-Agent Systems (MAS)ndash A system that consists of a number of agents

which interact with one-anotherndash Communication Coordination

Collaboration

23418 48

bull Ant Colony Optimization (ACO)ndash Inspired by ant colony foragingndash pheromone as heuristic information

(stigmergy)ndash Iteration between ConstructAntSolutions and

UpdatePheromones

bull Particle Swarm Optimization (PSO)ndash Inspired by bird flockingndash Heuristic information results from partnersndash Particle Velocity Update

• Collective Intelligence
• Outline
• The Computational Beauty of Nature
• Examples of Collective Behavior in Nature and Society
• Emergence
• Biological motivation Insect Societies
• Insect Societies
• Self Organization
• Stigmergy
• Flocks Herds and Schools
• Boid rules
• Slide 12
• Swarm Intelligence (Contrsquod)
• Components of SI
• Characteristics of SI
• The Web becomes a Giant Brain Some see the Web evolving into a collective brain for humankind
• What is Multi-agent Systems
• What is MAS(Contrsquod)
• Slide 19
• Communication models
• Working together
• Example mechanism Contract Net Protocol (CNP)
• Slide 23
• CNP (Contrsquod)
• Slide 25
• Attributes of Multi-agent Systems
• Applications of MAS
• Applications of MAS (Contrsquod)
• How to simulate SI for search
• Part Ⅱ Ant Colony Optimization (ACO)
• Natural Ants
• Slide 32
• How can the natural ants find the shortest path
• Slide 34
• Problems of AS
• Part Ⅲ Particle Swarm Optimization (PSO)
• Slide 37
• Modeling bird flocking
• Modeling bird flocking (Contrsquod)
• From Bird to Particle
• Features of PSO
• Particle Swarm Optimization Process
• How do particles fly
• Slide 44
• PSO Related issues
• Advantages of PSO
• Summary
• Slide 48

23418 21

Working together

bull Benevolent Agentsndash assume agents are benevolent our best

interest is their best interest

bull Self-Interested Agentsndash Agents will be assumed to act to further their

own interests possibly at expense of othersndash Potential for conflict

23418 22

Example mechanism Contract Net Protocol (CNP)

bull Negotiation as a collaboration mechanism

bull Negotiation on how tasks should be shared

ndash A task (plan) may be decomposed in a hierarchy of subtasks (hierarchical planning)

ndash An agent may subcontract another agent to perform a (sub)task

Contract

Bid

agent agent

CNP

Task announceme

nt

23418 24

CNP (Contrsquod)

Contractor

Potential candidate agents

Task announcement (broadcast)

Contractor

Candidate Candidate

Bid

Bid

Phase 1 Task Announcement

- The contractor agent publicly announces a task

- Potential candidates evaluate the task according to their won skills and availability

Phase 2 Submission of Bids Proposals

- Agents that satisfy the requiremenst ie are able to perform the task send their bid proposal to the contractor

23418 25

CNP (Contrsquod)

Contractor

Selected candidate

Contractor

Contracted agent

Contract

Phase 3 Selection

- The selection of the best candidate is made by the contractor based on received bids and on the CVs of the candidates

Phase 4 Contract awarding

- A contract is established between the contractor and the selected candidate

- A privileged bilateral communication channel is established between the two agents

23418 26

Attributes of Multi-agent Systems

Apply MAS when some of the following features show up in a problem

bull Decentralizationbull Complex components often best described at

the knowledge levelbull Adaptive behaviorbull Complex interactionsbull Coordinationbull Emergent aggregate behaviors

23418 27

Applications of MAS

Advanced Manufacturing Management Systemsminus Agents as representatives of machines users business

processes etc

Intelligent Information Search on Internetminus Some agents may show learning capabilities (learn the

preferences of their users )

Intelligent security enforcement on Internetminus Agents are representative of sensors or IDSs

Shopping Agents in Electronic Commerceminus With search price comparison and bargaining capabilities

23418 28

Applications of MAS (Contrsquod)

Multi-agent auction in E-commerce

Distributed Surveillanceminus For information search or to look for special events informing

their users of relevant news

Distributed Signal Processingminus For problem diagnosis situation assessment etc in the

network

Distributed Problem Solvingminus Collaborative design scheduling and planning

23418 29

How to simulate SI for search

Example1 Ant --gt Ant Colony Optimization (ACO)

Example2 Bird Flocking --gt Particle Swarm Optimization (PSO)

23418 30

Part Ant Colony Optimization Ⅱ(ACO)

First proposed by M Dorigo 1992

23418 31

Natural Antsbull Individual ants are simple insects with

limited memory and capable of performing simple actions

bull However an ant colony expresses a complex collective behavior providing intelligent solutions to problems such asndash carrying large itemsndash forming bridgesndash finding the shortest routes from the nest

to a food source prioritizing food sources based on their distance and ease of access

23418 32

Natural Antsbull Moreover in a colony each ant has its prescribed task but

the ants can switch tasks if the collective needs it ndash Outside the nest ants can have 4 different tasks

bull Foraging searching for and retrieving food

bull Patrolling looking for food supply

bull Midden work Sorting the colony refuse pile

bull Nest maintenance work construction and clearing of chambers

ndash An antrsquos decision whether to perform a task depends onbull The Phisical State of the environment

ndash If part of the nest is damaged more ants do nest maintenance work to repair it

bull Social Interactions with other ants

Communication (direct or indirect) is necessary

23418 33

bull They establish indirect communication system based on the deposition of pheromone over the path they followndash An isolated ant moves at random but when it finds a pheromone

trail there is a high probability that this ant will decide to follow the trail

ndash An ant foraging for food deposits pheromone over its route When it finds a food source it returns to the nest reinforcing its trail

ndash So other ants have greater probability to start following this trail and thereby laying more pheromone on it

ndash This process works as a positive feedback loop system because the higher the intensity of the pheromone over a trail the higher the probability of an ant start traveling through it

How can the natural ants find the shortest path

23418 34

bull Since the route B is shorter the ants on this path will complete the travel more times and thereby lay more pheromone over it

bull The pheromone concentration on trail B will increase at a higher rate than on A and soon the ants on route A will choose to follow route B

bull Since most ants will no longer travel on route A and since the pheromone is volatile trail A will start evaporating

bull Only the shortest route will remain

23418 35

Problems of ASbull Ant System tends to converge quickly

ndash This means that its exploitation of the best solution found is too high it should be exploring solution space more

ndash Pheromone evaporationupdate rule (better rule may exist)

bull Led to extensions of the ant systemndash Elitist Strategy for Ant Systems (EAS)ndash Rank based Ant Systems (AKRANK)ndash MAX-MIN Ant system (MMAS)ndash Ant Colony System (ACS)

23418 36

Part Ⅲ Particle Swarm Optimization (PSO)

1048713Firstly Proposed by Kennedy and Eberhart 1995

ldquoOnce again nature has provided us with a technique for processing information that is at once elegant and versatilerdquo

23418 37

bull In PSO a ldquoswarmrdquo is defined as an apparently disorganized collection (population) of moving individuals that tend to cluster together while each individual seems to be moving in a random direction

Bird flocking is one of the best example of PSO in nature

23418 38

Modeling bird flocking

bull The synchrony of flocking behavior is thought to be a function of birdrsquos efforts to maintain an optimal distance between themselves and their neighborsndash Birds and fish adjust their physical movement to avoid

predators seek for food and mates ndash Humans tend to adjust our beliefs and attitudes to

conform with those of our social peers Humans change in abstract multidimensional space collision-free

23418 39

Modeling bird flocking (Contrsquod)

bull Definitions ndash Flock is a group of objects that exhibit the general

class of polarized (aligned) non-colliding aggregate motion

ndash Boid is a simulated bird-like object ie it exhibits this type of behavior It can be a fish bee dinosaur etc

bull Rules for flockingndash Cohesion Each boid fly towards the centroid of its

local flock mates (that is boid in its local neighborhood)ndash Separation Each boid keep a certain distance away

from local flock mates to avoid collisionsndash Alignment Each boid align its velocity vector and keep

velocity magnitude similar with that of the local flock

23418 40

bull Imagine a birdrsquos flock in an area where there is a single food source

bull A bird donrsquot know where the food is but it knows its distance to the food

bull The best strategy is to follow the bird that is closer to the food

bull Particles save and communicate the best solution they have found

From Bird to Particle

23418 41

Features of PSO

bull Population initialized by assigning random positions and velocities potential solutions are then flown through hyperspace

bull Each particle keeps track of its ldquobestrdquo (highest fitness) position in hyperspacendash This is called ldquopBestrdquo for an individual particlendash It is called ldquogBestrdquo for the best in the populationndash It is called ldquolBestrdquo for the best in a defined neighborhood

bull At each time step each particle stochastically accelerates toward its pBest and gBest (or lBest)

23418 42

Particle Swarm Optimization Process

Step1 Initialize population in hyperspace

Step2 Evaluate fitness of individual particles

Step3 Modify velocities based on previous best and global (or neighborhood) best

Step4 Terminate on some condition

Step5 Go to step 2

23418 43

How do particles fly

bull Combination of gBest and the pBest (lBest)ndash need a compromise

bull lBest can bendash Social the particles around are always the same no matter

where they are in spacendash Geographical the particles around are those whose distance is

the shortest

bull Global PSO vs Local PSOndash the global version converges quickly to a solution but it gets

more easily stuck in local minima

23418 44

Illustrating the velocity update schema of global PSO

23418 45

PSO Related issues

bull Controlling velocities (determining the best value for Vmax)ndash Usually set maximum velocity to dynamic range of variable

bull Usually set c1 and c2 to 2

bull Inertia weight influences tradeoff between global and local explorationndash Good approach is to reduce inertia weight during run (ie from

09 to 04 over 1000 generations)

bull Swarm Size and Neighborhood Size

23418 46

Advantages of PSO

bull Adaptation operates on velocitiesndash Most other methods operate on positionsndash Effect PSO has a builtin momentumndash Particles tend to hurdle past optima ndash an advantage

since the best positions are remembered anyway

bull Simple in conceptbull Easy to implementbull Computationally efficientbull Effective on a variety of problems

23418 47

Summarybull Swarm Intelligence (SI)

ndash an artificial intelligence technique based around the study of collective behavior in decentralized self-organized systems

bull Multi-Agent Systems (MAS)ndash A system that consists of a number of agents

which interact with one-anotherndash Communication Coordination

Collaboration

23418 48

bull Ant Colony Optimization (ACO)ndash Inspired by ant colony foragingndash pheromone as heuristic information

(stigmergy)ndash Iteration between ConstructAntSolutions and

UpdatePheromones

bull Particle Swarm Optimization (PSO)ndash Inspired by bird flockingndash Heuristic information results from partnersndash Particle Velocity Update

• Collective Intelligence
• Outline
• The Computational Beauty of Nature
• Examples of Collective Behavior in Nature and Society
• Emergence
• Biological motivation Insect Societies
• Insect Societies
• Self Organization
• Stigmergy
• Flocks Herds and Schools
• Boid rules
• Slide 12
• Swarm Intelligence (Contrsquod)
• Components of SI
• Characteristics of SI
• The Web becomes a Giant Brain Some see the Web evolving into a collective brain for humankind
• What is Multi-agent Systems
• What is MAS(Contrsquod)
• Slide 19
• Communication models
• Working together
• Example mechanism Contract Net Protocol (CNP)
• Slide 23
• CNP (Contrsquod)
• Slide 25
• Attributes of Multi-agent Systems
• Applications of MAS
• Applications of MAS (Contrsquod)
• How to simulate SI for search
• Part Ⅱ Ant Colony Optimization (ACO)
• Natural Ants
• Slide 32
• How can the natural ants find the shortest path
• Slide 34
• Problems of AS
• Part Ⅲ Particle Swarm Optimization (PSO)
• Slide 37
• Modeling bird flocking
• Modeling bird flocking (Contrsquod)
• From Bird to Particle
• Features of PSO
• Particle Swarm Optimization Process
• How do particles fly
• Slide 44
• PSO Related issues
• Advantages of PSO
• Summary
• Slide 48

23418 22

Example mechanism Contract Net Protocol (CNP)

bull Negotiation as a collaboration mechanism

bull Negotiation on how tasks should be shared

ndash A task (plan) may be decomposed in a hierarchy of subtasks (hierarchical planning)

ndash An agent may subcontract another agent to perform a (sub)task

Contract

Bid

agent agent

CNP

Task announceme

nt

23418 24

CNP (Contrsquod)

Contractor

Potential candidate agents

Task announcement (broadcast)

Contractor

Candidate Candidate

Bid

Bid

Phase 1 Task Announcement

- The contractor agent publicly announces a task

- Potential candidates evaluate the task according to their won skills and availability

Phase 2 Submission of Bids Proposals

- Agents that satisfy the requiremenst ie are able to perform the task send their bid proposal to the contractor

23418 25

CNP (Contrsquod)

Contractor

Selected candidate

Contractor

Contracted agent

Contract

Phase 3 Selection

- The selection of the best candidate is made by the contractor based on received bids and on the CVs of the candidates

Phase 4 Contract awarding

- A contract is established between the contractor and the selected candidate

- A privileged bilateral communication channel is established between the two agents

23418 26

Attributes of Multi-agent Systems

Apply MAS when some of the following features show up in a problem

bull Decentralizationbull Complex components often best described at

the knowledge levelbull Adaptive behaviorbull Complex interactionsbull Coordinationbull Emergent aggregate behaviors

23418 27

Applications of MAS

Advanced Manufacturing Management Systemsminus Agents as representatives of machines users business

processes etc

Intelligent Information Search on Internetminus Some agents may show learning capabilities (learn the

preferences of their users )

Intelligent security enforcement on Internetminus Agents are representative of sensors or IDSs

Shopping Agents in Electronic Commerceminus With search price comparison and bargaining capabilities

23418 28

Applications of MAS (Contrsquod)

Multi-agent auction in E-commerce

Distributed Surveillanceminus For information search or to look for special events informing

their users of relevant news

Distributed Signal Processingminus For problem diagnosis situation assessment etc in the

network

Distributed Problem Solvingminus Collaborative design scheduling and planning

23418 29

How to simulate SI for search

Example1 Ant --gt Ant Colony Optimization (ACO)

Example2 Bird Flocking --gt Particle Swarm Optimization (PSO)

23418 30

Part Ant Colony Optimization Ⅱ(ACO)

First proposed by M Dorigo 1992

23418 31

Natural Antsbull Individual ants are simple insects with

limited memory and capable of performing simple actions

bull However an ant colony expresses a complex collective behavior providing intelligent solutions to problems such asndash carrying large itemsndash forming bridgesndash finding the shortest routes from the nest

to a food source prioritizing food sources based on their distance and ease of access

23418 32

Natural Antsbull Moreover in a colony each ant has its prescribed task but

the ants can switch tasks if the collective needs it ndash Outside the nest ants can have 4 different tasks

bull Foraging searching for and retrieving food

bull Patrolling looking for food supply

bull Midden work Sorting the colony refuse pile

bull Nest maintenance work construction and clearing of chambers

ndash An antrsquos decision whether to perform a task depends onbull The Phisical State of the environment

ndash If part of the nest is damaged more ants do nest maintenance work to repair it

bull Social Interactions with other ants

Communication (direct or indirect) is necessary

23418 33

bull They establish indirect communication system based on the deposition of pheromone over the path they followndash An isolated ant moves at random but when it finds a pheromone

trail there is a high probability that this ant will decide to follow the trail

ndash An ant foraging for food deposits pheromone over its route When it finds a food source it returns to the nest reinforcing its trail

ndash So other ants have greater probability to start following this trail and thereby laying more pheromone on it

ndash This process works as a positive feedback loop system because the higher the intensity of the pheromone over a trail the higher the probability of an ant start traveling through it

How can the natural ants find the shortest path

23418 34

bull Since the route B is shorter the ants on this path will complete the travel more times and thereby lay more pheromone over it

bull The pheromone concentration on trail B will increase at a higher rate than on A and soon the ants on route A will choose to follow route B

bull Since most ants will no longer travel on route A and since the pheromone is volatile trail A will start evaporating

bull Only the shortest route will remain

23418 35

Problems of ASbull Ant System tends to converge quickly

ndash This means that its exploitation of the best solution found is too high it should be exploring solution space more

ndash Pheromone evaporationupdate rule (better rule may exist)

bull Led to extensions of the ant systemndash Elitist Strategy for Ant Systems (EAS)ndash Rank based Ant Systems (AKRANK)ndash MAX-MIN Ant system (MMAS)ndash Ant Colony System (ACS)

23418 36

Part Ⅲ Particle Swarm Optimization (PSO)

1048713Firstly Proposed by Kennedy and Eberhart 1995

ldquoOnce again nature has provided us with a technique for processing information that is at once elegant and versatilerdquo

23418 37

bull In PSO a ldquoswarmrdquo is defined as an apparently disorganized collection (population) of moving individuals that tend to cluster together while each individual seems to be moving in a random direction

Bird flocking is one of the best example of PSO in nature

23418 38

Modeling bird flocking

bull The synchrony of flocking behavior is thought to be a function of birdrsquos efforts to maintain an optimal distance between themselves and their neighborsndash Birds and fish adjust their physical movement to avoid

predators seek for food and mates ndash Humans tend to adjust our beliefs and attitudes to

conform with those of our social peers Humans change in abstract multidimensional space collision-free

23418 39

Modeling bird flocking (Contrsquod)

bull Definitions ndash Flock is a group of objects that exhibit the general

class of polarized (aligned) non-colliding aggregate motion

ndash Boid is a simulated bird-like object ie it exhibits this type of behavior It can be a fish bee dinosaur etc

bull Rules for flockingndash Cohesion Each boid fly towards the centroid of its

local flock mates (that is boid in its local neighborhood)ndash Separation Each boid keep a certain distance away

from local flock mates to avoid collisionsndash Alignment Each boid align its velocity vector and keep

velocity magnitude similar with that of the local flock

23418 40

bull Imagine a birdrsquos flock in an area where there is a single food source

bull A bird donrsquot know where the food is but it knows its distance to the food

bull The best strategy is to follow the bird that is closer to the food

bull Particles save and communicate the best solution they have found

From Bird to Particle

23418 41

Features of PSO

bull Population initialized by assigning random positions and velocities potential solutions are then flown through hyperspace

bull Each particle keeps track of its ldquobestrdquo (highest fitness) position in hyperspacendash This is called ldquopBestrdquo for an individual particlendash It is called ldquogBestrdquo for the best in the populationndash It is called ldquolBestrdquo for the best in a defined neighborhood

bull At each time step each particle stochastically accelerates toward its pBest and gBest (or lBest)

23418 42

Particle Swarm Optimization Process

Step1 Initialize population in hyperspace

Step2 Evaluate fitness of individual particles

Step3 Modify velocities based on previous best and global (or neighborhood) best

Step4 Terminate on some condition

Step5 Go to step 2

23418 43

How do particles fly

bull Combination of gBest and the pBest (lBest)ndash need a compromise

bull lBest can bendash Social the particles around are always the same no matter

where they are in spacendash Geographical the particles around are those whose distance is

the shortest

bull Global PSO vs Local PSOndash the global version converges quickly to a solution but it gets

more easily stuck in local minima

23418 44

Illustrating the velocity update schema of global PSO

23418 45

PSO Related issues

bull Controlling velocities (determining the best value for Vmax)ndash Usually set maximum velocity to dynamic range of variable

bull Usually set c1 and c2 to 2

bull Inertia weight influences tradeoff between global and local explorationndash Good approach is to reduce inertia weight during run (ie from

09 to 04 over 1000 generations)

bull Swarm Size and Neighborhood Size

23418 46

Advantages of PSO

bull Adaptation operates on velocitiesndash Most other methods operate on positionsndash Effect PSO has a builtin momentumndash Particles tend to hurdle past optima ndash an advantage

since the best positions are remembered anyway

bull Simple in conceptbull Easy to implementbull Computationally efficientbull Effective on a variety of problems

23418 47

Summarybull Swarm Intelligence (SI)

ndash an artificial intelligence technique based around the study of collective behavior in decentralized self-organized systems

bull Multi-Agent Systems (MAS)ndash A system that consists of a number of agents

which interact with one-anotherndash Communication Coordination

Collaboration

23418 48

bull Ant Colony Optimization (ACO)ndash Inspired by ant colony foragingndash pheromone as heuristic information

(stigmergy)ndash Iteration between ConstructAntSolutions and

UpdatePheromones

bull Particle Swarm Optimization (PSO)ndash Inspired by bird flockingndash Heuristic information results from partnersndash Particle Velocity Update

• Collective Intelligence
• Outline
• The Computational Beauty of Nature
• Examples of Collective Behavior in Nature and Society
• Emergence
• Biological motivation Insect Societies
• Insect Societies
• Self Organization
• Stigmergy
• Flocks Herds and Schools
• Boid rules
• Slide 12
• Swarm Intelligence (Contrsquod)
• Components of SI
• Characteristics of SI
• The Web becomes a Giant Brain Some see the Web evolving into a collective brain for humankind
• What is Multi-agent Systems
• What is MAS(Contrsquod)
• Slide 19
• Communication models
• Working together
• Example mechanism Contract Net Protocol (CNP)
• Slide 23
• CNP (Contrsquod)
• Slide 25
• Attributes of Multi-agent Systems
• Applications of MAS
• Applications of MAS (Contrsquod)
• How to simulate SI for search
• Part Ⅱ Ant Colony Optimization (ACO)
• Natural Ants
• Slide 32
• How can the natural ants find the shortest path
• Slide 34
• Problems of AS
• Part Ⅲ Particle Swarm Optimization (PSO)
• Slide 37
• Modeling bird flocking
• Modeling bird flocking (Contrsquod)
• From Bird to Particle
• Features of PSO
• Particle Swarm Optimization Process
• How do particles fly
• Slide 44
• PSO Related issues
• Advantages of PSO
• Summary
• Slide 48

Contract

Bid

agent agent

CNP

Task announceme

nt

23418 24

CNP (Contrsquod)

Contractor

Potential candidate agents

Task announcement (broadcast)

Contractor

Candidate Candidate

Bid

Bid

Phase 1 Task Announcement

- The contractor agent publicly announces a task

- Potential candidates evaluate the task according to their won skills and availability

Phase 2 Submission of Bids Proposals

- Agents that satisfy the requiremenst ie are able to perform the task send their bid proposal to the contractor

23418 25

CNP (Contrsquod)

Contractor

Selected candidate

Contractor

Contracted agent

Contract

Phase 3 Selection

- The selection of the best candidate is made by the contractor based on received bids and on the CVs of the candidates

Phase 4 Contract awarding

- A contract is established between the contractor and the selected candidate

- A privileged bilateral communication channel is established between the two agents

23418 26

Attributes of Multi-agent Systems

Apply MAS when some of the following features show up in a problem

bull Decentralizationbull Complex components often best described at

the knowledge levelbull Adaptive behaviorbull Complex interactionsbull Coordinationbull Emergent aggregate behaviors

23418 27

Applications of MAS

Advanced Manufacturing Management Systemsminus Agents as representatives of machines users business

processes etc

Intelligent Information Search on Internetminus Some agents may show learning capabilities (learn the

preferences of their users )

Intelligent security enforcement on Internetminus Agents are representative of sensors or IDSs

Shopping Agents in Electronic Commerceminus With search price comparison and bargaining capabilities

23418 28

Applications of MAS (Contrsquod)

Multi-agent auction in E-commerce

Distributed Surveillanceminus For information search or to look for special events informing

their users of relevant news

Distributed Signal Processingminus For problem diagnosis situation assessment etc in the

network

Distributed Problem Solvingminus Collaborative design scheduling and planning

23418 29

How to simulate SI for search

Example1 Ant --gt Ant Colony Optimization (ACO)

Example2 Bird Flocking --gt Particle Swarm Optimization (PSO)

23418 30

Part Ant Colony Optimization Ⅱ(ACO)

First proposed by M Dorigo 1992

23418 31

Natural Antsbull Individual ants are simple insects with

limited memory and capable of performing simple actions

bull However an ant colony expresses a complex collective behavior providing intelligent solutions to problems such asndash carrying large itemsndash forming bridgesndash finding the shortest routes from the nest

to a food source prioritizing food sources based on their distance and ease of access

23418 32

Natural Antsbull Moreover in a colony each ant has its prescribed task but

the ants can switch tasks if the collective needs it ndash Outside the nest ants can have 4 different tasks

bull Foraging searching for and retrieving food

bull Patrolling looking for food supply

bull Midden work Sorting the colony refuse pile

bull Nest maintenance work construction and clearing of chambers

ndash An antrsquos decision whether to perform a task depends onbull The Phisical State of the environment

ndash If part of the nest is damaged more ants do nest maintenance work to repair it

bull Social Interactions with other ants

Communication (direct or indirect) is necessary

23418 33

bull They establish indirect communication system based on the deposition of pheromone over the path they followndash An isolated ant moves at random but when it finds a pheromone

trail there is a high probability that this ant will decide to follow the trail

ndash An ant foraging for food deposits pheromone over its route When it finds a food source it returns to the nest reinforcing its trail

ndash So other ants have greater probability to start following this trail and thereby laying more pheromone on it

ndash This process works as a positive feedback loop system because the higher the intensity of the pheromone over a trail the higher the probability of an ant start traveling through it

How can the natural ants find the shortest path

23418 34

bull Since the route B is shorter the ants on this path will complete the travel more times and thereby lay more pheromone over it

bull The pheromone concentration on trail B will increase at a higher rate than on A and soon the ants on route A will choose to follow route B

bull Since most ants will no longer travel on route A and since the pheromone is volatile trail A will start evaporating

bull Only the shortest route will remain

23418 35

Problems of ASbull Ant System tends to converge quickly

ndash This means that its exploitation of the best solution found is too high it should be exploring solution space more

ndash Pheromone evaporationupdate rule (better rule may exist)

bull Led to extensions of the ant systemndash Elitist Strategy for Ant Systems (EAS)ndash Rank based Ant Systems (AKRANK)ndash MAX-MIN Ant system (MMAS)ndash Ant Colony System (ACS)

23418 36

Part Ⅲ Particle Swarm Optimization (PSO)

1048713Firstly Proposed by Kennedy and Eberhart 1995

ldquoOnce again nature has provided us with a technique for processing information that is at once elegant and versatilerdquo

23418 37

bull In PSO a ldquoswarmrdquo is defined as an apparently disorganized collection (population) of moving individuals that tend to cluster together while each individual seems to be moving in a random direction

Bird flocking is one of the best example of PSO in nature

23418 38

Modeling bird flocking

bull The synchrony of flocking behavior is thought to be a function of birdrsquos efforts to maintain an optimal distance between themselves and their neighborsndash Birds and fish adjust their physical movement to avoid

predators seek for food and mates ndash Humans tend to adjust our beliefs and attitudes to

conform with those of our social peers Humans change in abstract multidimensional space collision-free

23418 39

Modeling bird flocking (Contrsquod)

bull Definitions ndash Flock is a group of objects that exhibit the general

class of polarized (aligned) non-colliding aggregate motion

ndash Boid is a simulated bird-like object ie it exhibits this type of behavior It can be a fish bee dinosaur etc

bull Rules for flockingndash Cohesion Each boid fly towards the centroid of its

local flock mates (that is boid in its local neighborhood)ndash Separation Each boid keep a certain distance away

from local flock mates to avoid collisionsndash Alignment Each boid align its velocity vector and keep

velocity magnitude similar with that of the local flock

23418 40

bull Imagine a birdrsquos flock in an area where there is a single food source

bull A bird donrsquot know where the food is but it knows its distance to the food

bull The best strategy is to follow the bird that is closer to the food

bull Particles save and communicate the best solution they have found

From Bird to Particle

23418 41

Features of PSO

bull Population initialized by assigning random positions and velocities potential solutions are then flown through hyperspace

bull Each particle keeps track of its ldquobestrdquo (highest fitness) position in hyperspacendash This is called ldquopBestrdquo for an individual particlendash It is called ldquogBestrdquo for the best in the populationndash It is called ldquolBestrdquo for the best in a defined neighborhood

bull At each time step each particle stochastically accelerates toward its pBest and gBest (or lBest)

23418 42

Particle Swarm Optimization Process

Step1 Initialize population in hyperspace

Step2 Evaluate fitness of individual particles

Step3 Modify velocities based on previous best and global (or neighborhood) best

Step4 Terminate on some condition

Step5 Go to step 2

23418 43

How do particles fly

bull Combination of gBest and the pBest (lBest)ndash need a compromise

bull lBest can bendash Social the particles around are always the same no matter

where they are in spacendash Geographical the particles around are those whose distance is

the shortest

bull Global PSO vs Local PSOndash the global version converges quickly to a solution but it gets

more easily stuck in local minima

23418 44

Illustrating the velocity update schema of global PSO

23418 45

PSO Related issues

bull Controlling velocities (determining the best value for Vmax)ndash Usually set maximum velocity to dynamic range of variable

bull Usually set c1 and c2 to 2

bull Inertia weight influences tradeoff between global and local explorationndash Good approach is to reduce inertia weight during run (ie from

09 to 04 over 1000 generations)

bull Swarm Size and Neighborhood Size

23418 46

Advantages of PSO

bull Adaptation operates on velocitiesndash Most other methods operate on positionsndash Effect PSO has a builtin momentumndash Particles tend to hurdle past optima ndash an advantage

since the best positions are remembered anyway

bull Simple in conceptbull Easy to implementbull Computationally efficientbull Effective on a variety of problems

23418 47

Summarybull Swarm Intelligence (SI)

ndash an artificial intelligence technique based around the study of collective behavior in decentralized self-organized systems

bull Multi-Agent Systems (MAS)ndash A system that consists of a number of agents

which interact with one-anotherndash Communication Coordination

Collaboration

23418 48

bull Ant Colony Optimization (ACO)ndash Inspired by ant colony foragingndash pheromone as heuristic information

(stigmergy)ndash Iteration between ConstructAntSolutions and

UpdatePheromones

bull Particle Swarm Optimization (PSO)ndash Inspired by bird flockingndash Heuristic information results from partnersndash Particle Velocity Update

• Collective Intelligence
• Outline
• The Computational Beauty of Nature
• Examples of Collective Behavior in Nature and Society
• Emergence
• Biological motivation Insect Societies
• Insect Societies
• Self Organization
• Stigmergy
• Flocks Herds and Schools
• Boid rules
• Slide 12
• Swarm Intelligence (Contrsquod)
• Components of SI
• Characteristics of SI
• The Web becomes a Giant Brain Some see the Web evolving into a collective brain for humankind
• What is Multi-agent Systems
• What is MAS(Contrsquod)
• Slide 19
• Communication models
• Working together
• Example mechanism Contract Net Protocol (CNP)
• Slide 23
• CNP (Contrsquod)
• Slide 25
• Attributes of Multi-agent Systems
• Applications of MAS
• Applications of MAS (Contrsquod)
• How to simulate SI for search
• Part Ⅱ Ant Colony Optimization (ACO)
• Natural Ants
• Slide 32
• How can the natural ants find the shortest path
• Slide 34
• Problems of AS
• Part Ⅲ Particle Swarm Optimization (PSO)
• Slide 37
• Modeling bird flocking
• Modeling bird flocking (Contrsquod)
• From Bird to Particle
• Features of PSO
• Particle Swarm Optimization Process
• How do particles fly
• Slide 44
• PSO Related issues
• Advantages of PSO
• Summary
• Slide 48

23418 24

CNP (Contrsquod)

Contractor

Potential candidate agents

Task announcement (broadcast)

Contractor

Candidate Candidate

Bid

Bid

Phase 1 Task Announcement

- The contractor agent publicly announces a task

- Potential candidates evaluate the task according to their won skills and availability

Phase 2 Submission of Bids Proposals

- Agents that satisfy the requiremenst ie are able to perform the task send their bid proposal to the contractor

23418 25

CNP (Contrsquod)

Contractor

Selected candidate

Contractor

Contracted agent

Contract

Phase 3 Selection

- The selection of the best candidate is made by the contractor based on received bids and on the CVs of the candidates

Phase 4 Contract awarding

- A contract is established between the contractor and the selected candidate

- A privileged bilateral communication channel is established between the two agents

23418 26

Attributes of Multi-agent Systems

Apply MAS when some of the following features show up in a problem

bull Decentralizationbull Complex components often best described at

the knowledge levelbull Adaptive behaviorbull Complex interactionsbull Coordinationbull Emergent aggregate behaviors

23418 27

Applications of MAS

Advanced Manufacturing Management Systemsminus Agents as representatives of machines users business

processes etc

Intelligent Information Search on Internetminus Some agents may show learning capabilities (learn the

preferences of their users )

Intelligent security enforcement on Internetminus Agents are representative of sensors or IDSs

Shopping Agents in Electronic Commerceminus With search price comparison and bargaining capabilities

23418 28

Applications of MAS (Contrsquod)

Multi-agent auction in E-commerce

Distributed Surveillanceminus For information search or to look for special events informing

their users of relevant news

Distributed Signal Processingminus For problem diagnosis situation assessment etc in the

network

Distributed Problem Solvingminus Collaborative design scheduling and planning

23418 29

How to simulate SI for search

Example1 Ant --gt Ant Colony Optimization (ACO)

Example2 Bird Flocking --gt Particle Swarm Optimization (PSO)

23418 30

Part Ant Colony Optimization Ⅱ(ACO)

First proposed by M Dorigo 1992

23418 31

Natural Antsbull Individual ants are simple insects with

limited memory and capable of performing simple actions

bull However an ant colony expresses a complex collective behavior providing intelligent solutions to problems such asndash carrying large itemsndash forming bridgesndash finding the shortest routes from the nest

to a food source prioritizing food sources based on their distance and ease of access

23418 32

Natural Antsbull Moreover in a colony each ant has its prescribed task but

the ants can switch tasks if the collective needs it ndash Outside the nest ants can have 4 different tasks

bull Foraging searching for and retrieving food

bull Patrolling looking for food supply

bull Midden work Sorting the colony refuse pile

bull Nest maintenance work construction and clearing of chambers

ndash An antrsquos decision whether to perform a task depends onbull The Phisical State of the environment

ndash If part of the nest is damaged more ants do nest maintenance work to repair it

bull Social Interactions with other ants

Communication (direct or indirect) is necessary

23418 33

bull They establish indirect communication system based on the deposition of pheromone over the path they followndash An isolated ant moves at random but when it finds a pheromone

trail there is a high probability that this ant will decide to follow the trail

ndash An ant foraging for food deposits pheromone over its route When it finds a food source it returns to the nest reinforcing its trail

ndash So other ants have greater probability to start following this trail and thereby laying more pheromone on it

ndash This process works as a positive feedback loop system because the higher the intensity of the pheromone over a trail the higher the probability of an ant start traveling through it

How can the natural ants find the shortest path

23418 34

bull Since the route B is shorter the ants on this path will complete the travel more times and thereby lay more pheromone over it

bull The pheromone concentration on trail B will increase at a higher rate than on A and soon the ants on route A will choose to follow route B

bull Since most ants will no longer travel on route A and since the pheromone is volatile trail A will start evaporating

bull Only the shortest route will remain

23418 35

Problems of ASbull Ant System tends to converge quickly

ndash This means that its exploitation of the best solution found is too high it should be exploring solution space more

ndash Pheromone evaporationupdate rule (better rule may exist)

bull Led to extensions of the ant systemndash Elitist Strategy for Ant Systems (EAS)ndash Rank based Ant Systems (AKRANK)ndash MAX-MIN Ant system (MMAS)ndash Ant Colony System (ACS)

23418 36

Part Ⅲ Particle Swarm Optimization (PSO)

1048713Firstly Proposed by Kennedy and Eberhart 1995

ldquoOnce again nature has provided us with a technique for processing information that is at once elegant and versatilerdquo

23418 37

bull In PSO a ldquoswarmrdquo is defined as an apparently disorganized collection (population) of moving individuals that tend to cluster together while each individual seems to be moving in a random direction

Bird flocking is one of the best example of PSO in nature

23418 38

Modeling bird flocking

bull The synchrony of flocking behavior is thought to be a function of birdrsquos efforts to maintain an optimal distance between themselves and their neighborsndash Birds and fish adjust their physical movement to avoid

predators seek for food and mates ndash Humans tend to adjust our beliefs and attitudes to

conform with those of our social peers Humans change in abstract multidimensional space collision-free

23418 39

Modeling bird flocking (Contrsquod)

bull Definitions ndash Flock is a group of objects that exhibit the general

class of polarized (aligned) non-colliding aggregate motion

ndash Boid is a simulated bird-like object ie it exhibits this type of behavior It can be a fish bee dinosaur etc

bull Rules for flockingndash Cohesion Each boid fly towards the centroid of its

local flock mates (that is boid in its local neighborhood)ndash Separation Each boid keep a certain distance away

from local flock mates to avoid collisionsndash Alignment Each boid align its velocity vector and keep

velocity magnitude similar with that of the local flock

23418 40

bull Imagine a birdrsquos flock in an area where there is a single food source

bull A bird donrsquot know where the food is but it knows its distance to the food

bull The best strategy is to follow the bird that is closer to the food

bull Particles save and communicate the best solution they have found

From Bird to Particle

23418 41

Features of PSO

bull Population initialized by assigning random positions and velocities potential solutions are then flown through hyperspace

bull Each particle keeps track of its ldquobestrdquo (highest fitness) position in hyperspacendash This is called ldquopBestrdquo for an individual particlendash It is called ldquogBestrdquo for the best in the populationndash It is called ldquolBestrdquo for the best in a defined neighborhood

bull At each time step each particle stochastically accelerates toward its pBest and gBest (or lBest)

23418 42

Particle Swarm Optimization Process

Step1 Initialize population in hyperspace

Step2 Evaluate fitness of individual particles

Step3 Modify velocities based on previous best and global (or neighborhood) best

Step4 Terminate on some condition

Step5 Go to step 2

23418 43

How do particles fly

bull Combination of gBest and the pBest (lBest)ndash need a compromise

bull lBest can bendash Social the particles around are always the same no matter

where they are in spacendash Geographical the particles around are those whose distance is

the shortest

bull Global PSO vs Local PSOndash the global version converges quickly to a solution but it gets

more easily stuck in local minima

23418 44

Illustrating the velocity update schema of global PSO

23418 45

PSO Related issues

bull Controlling velocities (determining the best value for Vmax)ndash Usually set maximum velocity to dynamic range of variable

bull Usually set c1 and c2 to 2

bull Inertia weight influences tradeoff between global and local explorationndash Good approach is to reduce inertia weight during run (ie from

09 to 04 over 1000 generations)

bull Swarm Size and Neighborhood Size

23418 46

Advantages of PSO

bull Adaptation operates on velocitiesndash Most other methods operate on positionsndash Effect PSO has a builtin momentumndash Particles tend to hurdle past optima ndash an advantage

since the best positions are remembered anyway

bull Simple in conceptbull Easy to implementbull Computationally efficientbull Effective on a variety of problems

23418 47

Summarybull Swarm Intelligence (SI)

ndash an artificial intelligence technique based around the study of collective behavior in decentralized self-organized systems

bull Multi-Agent Systems (MAS)ndash A system that consists of a number of agents

which interact with one-anotherndash Communication Coordination

Collaboration

23418 48

bull Ant Colony Optimization (ACO)ndash Inspired by ant colony foragingndash pheromone as heuristic information

(stigmergy)ndash Iteration between ConstructAntSolutions and

UpdatePheromones

bull Particle Swarm Optimization (PSO)ndash Inspired by bird flockingndash Heuristic information results from partnersndash Particle Velocity Update

• Collective Intelligence
• Outline
• The Computational Beauty of Nature
• Examples of Collective Behavior in Nature and Society
• Emergence
• Biological motivation Insect Societies
• Insect Societies
• Self Organization
• Stigmergy
• Flocks Herds and Schools
• Boid rules
• Slide 12
• Swarm Intelligence (Contrsquod)
• Components of SI
• Characteristics of SI
• The Web becomes a Giant Brain Some see the Web evolving into a collective brain for humankind
• What is Multi-agent Systems
• What is MAS(Contrsquod)
• Slide 19
• Communication models
• Working together
• Example mechanism Contract Net Protocol (CNP)
• Slide 23
• CNP (Contrsquod)
• Slide 25
• Attributes of Multi-agent Systems
• Applications of MAS
• Applications of MAS (Contrsquod)
• How to simulate SI for search
• Part Ⅱ Ant Colony Optimization (ACO)
• Natural Ants
• Slide 32
• How can the natural ants find the shortest path
• Slide 34
• Problems of AS
• Part Ⅲ Particle Swarm Optimization (PSO)
• Slide 37
• Modeling bird flocking
• Modeling bird flocking (Contrsquod)
• From Bird to Particle
• Features of PSO
• Particle Swarm Optimization Process
• How do particles fly
• Slide 44
• PSO Related issues
• Advantages of PSO
• Summary
• Slide 48

23418 25

CNP (Contrsquod)

Contractor

Selected candidate

Contractor

Contracted agent

Contract

Phase 3 Selection

- The selection of the best candidate is made by the contractor based on received bids and on the CVs of the candidates

Phase 4 Contract awarding

- A contract is established between the contractor and the selected candidate

- A privileged bilateral communication channel is established between the two agents

23418 26

Attributes of Multi-agent Systems

Apply MAS when some of the following features show up in a problem

bull Decentralizationbull Complex components often best described at

the knowledge levelbull Adaptive behaviorbull Complex interactionsbull Coordinationbull Emergent aggregate behaviors

23418 27

Applications of MAS

Advanced Manufacturing Management Systemsminus Agents as representatives of machines users business

processes etc

Intelligent Information Search on Internetminus Some agents may show learning capabilities (learn the

preferences of their users )

Intelligent security enforcement on Internetminus Agents are representative of sensors or IDSs

Shopping Agents in Electronic Commerceminus With search price comparison and bargaining capabilities

23418 28

Applications of MAS (Contrsquod)

Multi-agent auction in E-commerce

Distributed Surveillanceminus For information search or to look for special events informing

their users of relevant news

Distributed Signal Processingminus For problem diagnosis situation assessment etc in the

network

Distributed Problem Solvingminus Collaborative design scheduling and planning

23418 29

How to simulate SI for search

Example1 Ant --gt Ant Colony Optimization (ACO)

Example2 Bird Flocking --gt Particle Swarm Optimization (PSO)

23418 30

Part Ant Colony Optimization Ⅱ(ACO)

First proposed by M Dorigo 1992

23418 31

Natural Antsbull Individual ants are simple insects with

limited memory and capable of performing simple actions

bull However an ant colony expresses a complex collective behavior providing intelligent solutions to problems such asndash carrying large itemsndash forming bridgesndash finding the shortest routes from the nest

to a food source prioritizing food sources based on their distance and ease of access

23418 32

Natural Antsbull Moreover in a colony each ant has its prescribed task but

the ants can switch tasks if the collective needs it ndash Outside the nest ants can have 4 different tasks

bull Foraging searching for and retrieving food

bull Patrolling looking for food supply

bull Midden work Sorting the colony refuse pile

bull Nest maintenance work construction and clearing of chambers

ndash An antrsquos decision whether to perform a task depends onbull The Phisical State of the environment

ndash If part of the nest is damaged more ants do nest maintenance work to repair it

bull Social Interactions with other ants

Communication (direct or indirect) is necessary

23418 33

bull They establish indirect communication system based on the deposition of pheromone over the path they followndash An isolated ant moves at random but when it finds a pheromone

trail there is a high probability that this ant will decide to follow the trail

ndash An ant foraging for food deposits pheromone over its route When it finds a food source it returns to the nest reinforcing its trail

ndash So other ants have greater probability to start following this trail and thereby laying more pheromone on it

ndash This process works as a positive feedback loop system because the higher the intensity of the pheromone over a trail the higher the probability of an ant start traveling through it

How can the natural ants find the shortest path

23418 34

bull Since the route B is shorter the ants on this path will complete the travel more times and thereby lay more pheromone over it

bull The pheromone concentration on trail B will increase at a higher rate than on A and soon the ants on route A will choose to follow route B

bull Since most ants will no longer travel on route A and since the pheromone is volatile trail A will start evaporating

bull Only the shortest route will remain

23418 35

Problems of ASbull Ant System tends to converge quickly

ndash This means that its exploitation of the best solution found is too high it should be exploring solution space more

ndash Pheromone evaporationupdate rule (better rule may exist)

bull Led to extensions of the ant systemndash Elitist Strategy for Ant Systems (EAS)ndash Rank based Ant Systems (AKRANK)ndash MAX-MIN Ant system (MMAS)ndash Ant Colony System (ACS)

23418 36

Part Ⅲ Particle Swarm Optimization (PSO)

1048713Firstly Proposed by Kennedy and Eberhart 1995

ldquoOnce again nature has provided us with a technique for processing information that is at once elegant and versatilerdquo

23418 37

bull In PSO a ldquoswarmrdquo is defined as an apparently disorganized collection (population) of moving individuals that tend to cluster together while each individual seems to be moving in a random direction

Bird flocking is one of the best example of PSO in nature

23418 38

Modeling bird flocking

bull The synchrony of flocking behavior is thought to be a function of birdrsquos efforts to maintain an optimal distance between themselves and their neighborsndash Birds and fish adjust their physical movement to avoid

predators seek for food and mates ndash Humans tend to adjust our beliefs and attitudes to

conform with those of our social peers Humans change in abstract multidimensional space collision-free

23418 39

Modeling bird flocking (Contrsquod)

bull Definitions ndash Flock is a group of objects that exhibit the general

class of polarized (aligned) non-colliding aggregate motion

ndash Boid is a simulated bird-like object ie it exhibits this type of behavior It can be a fish bee dinosaur etc

bull Rules for flockingndash Cohesion Each boid fly towards the centroid of its

local flock mates (that is boid in its local neighborhood)ndash Separation Each boid keep a certain distance away

from local flock mates to avoid collisionsndash Alignment Each boid align its velocity vector and keep

velocity magnitude similar with that of the local flock

23418 40

bull Imagine a birdrsquos flock in an area where there is a single food source

bull A bird donrsquot know where the food is but it knows its distance to the food

bull The best strategy is to follow the bird that is closer to the food

bull Particles save and communicate the best solution they have found

From Bird to Particle

23418 41

Features of PSO

bull Population initialized by assigning random positions and velocities potential solutions are then flown through hyperspace

bull Each particle keeps track of its ldquobestrdquo (highest fitness) position in hyperspacendash This is called ldquopBestrdquo for an individual particlendash It is called ldquogBestrdquo for the best in the populationndash It is called ldquolBestrdquo for the best in a defined neighborhood

bull At each time step each particle stochastically accelerates toward its pBest and gBest (or lBest)

23418 42

Particle Swarm Optimization Process

Step1 Initialize population in hyperspace

Step2 Evaluate fitness of individual particles

Step3 Modify velocities based on previous best and global (or neighborhood) best

Step4 Terminate on some condition

Step5 Go to step 2

23418 43

How do particles fly

bull Combination of gBest and the pBest (lBest)ndash need a compromise

bull lBest can bendash Social the particles around are always the same no matter

where they are in spacendash Geographical the particles around are those whose distance is

the shortest

bull Global PSO vs Local PSOndash the global version converges quickly to a solution but it gets

more easily stuck in local minima

23418 44

Illustrating the velocity update schema of global PSO

23418 45

PSO Related issues

bull Controlling velocities (determining the best value for Vmax)ndash Usually set maximum velocity to dynamic range of variable

bull Usually set c1 and c2 to 2

bull Inertia weight influences tradeoff between global and local explorationndash Good approach is to reduce inertia weight during run (ie from

09 to 04 over 1000 generations)

bull Swarm Size and Neighborhood Size

23418 46

Advantages of PSO

bull Adaptation operates on velocitiesndash Most other methods operate on positionsndash Effect PSO has a builtin momentumndash Particles tend to hurdle past optima ndash an advantage

since the best positions are remembered anyway

bull Simple in conceptbull Easy to implementbull Computationally efficientbull Effective on a variety of problems

23418 47

Summarybull Swarm Intelligence (SI)

ndash an artificial intelligence technique based around the study of collective behavior in decentralized self-organized systems

bull Multi-Agent Systems (MAS)ndash A system that consists of a number of agents

which interact with one-anotherndash Communication Coordination

Collaboration

23418 48

bull Ant Colony Optimization (ACO)ndash Inspired by ant colony foragingndash pheromone as heuristic information

(stigmergy)ndash Iteration between ConstructAntSolutions and

UpdatePheromones

bull Particle Swarm Optimization (PSO)ndash Inspired by bird flockingndash Heuristic information results from partnersndash Particle Velocity Update

• Collective Intelligence
• Outline
• The Computational Beauty of Nature
• Examples of Collective Behavior in Nature and Society
• Emergence
• Biological motivation Insect Societies
• Insect Societies
• Self Organization
• Stigmergy
• Flocks Herds and Schools
• Boid rules
• Slide 12
• Swarm Intelligence (Contrsquod)
• Components of SI
• Characteristics of SI
• The Web becomes a Giant Brain Some see the Web evolving into a collective brain for humankind
• What is Multi-agent Systems
• What is MAS(Contrsquod)
• Slide 19
• Communication models
• Working together
• Example mechanism Contract Net Protocol (CNP)
• Slide 23
• CNP (Contrsquod)
• Slide 25
• Attributes of Multi-agent Systems
• Applications of MAS
• Applications of MAS (Contrsquod)
• How to simulate SI for search
• Part Ⅱ Ant Colony Optimization (ACO)
• Natural Ants
• Slide 32
• How can the natural ants find the shortest path
• Slide 34
• Problems of AS
• Part Ⅲ Particle Swarm Optimization (PSO)
• Slide 37
• Modeling bird flocking
• Modeling bird flocking (Contrsquod)
• From Bird to Particle
• Features of PSO
• Particle Swarm Optimization Process
• How do particles fly
• Slide 44
• PSO Related issues
• Advantages of PSO
• Summary
• Slide 48

23418 26

Attributes of Multi-agent Systems

Apply MAS when some of the following features show up in a problem

bull Decentralizationbull Complex components often best described at

the knowledge levelbull Adaptive behaviorbull Complex interactionsbull Coordinationbull Emergent aggregate behaviors

23418 27

Applications of MAS

Advanced Manufacturing Management Systemsminus Agents as representatives of machines users business

processes etc

Intelligent Information Search on Internetminus Some agents may show learning capabilities (learn the

preferences of their users )

Intelligent security enforcement on Internetminus Agents are representative of sensors or IDSs

Shopping Agents in Electronic Commerceminus With search price comparison and bargaining capabilities

23418 28

Applications of MAS (Contrsquod)

Multi-agent auction in E-commerce

Distributed Surveillanceminus For information search or to look for special events informing

their users of relevant news

Distributed Signal Processingminus For problem diagnosis situation assessment etc in the

network

Distributed Problem Solvingminus Collaborative design scheduling and planning

23418 29

How to simulate SI for search

Example1 Ant --gt Ant Colony Optimization (ACO)

Example2 Bird Flocking --gt Particle Swarm Optimization (PSO)

23418 30

Part Ant Colony Optimization Ⅱ(ACO)

First proposed by M Dorigo 1992

23418 31

Natural Antsbull Individual ants are simple insects with

limited memory and capable of performing simple actions

bull However an ant colony expresses a complex collective behavior providing intelligent solutions to problems such asndash carrying large itemsndash forming bridgesndash finding the shortest routes from the nest

to a food source prioritizing food sources based on their distance and ease of access

23418 32

Natural Antsbull Moreover in a colony each ant has its prescribed task but

the ants can switch tasks if the collective needs it ndash Outside the nest ants can have 4 different tasks

bull Foraging searching for and retrieving food

bull Patrolling looking for food supply

bull Midden work Sorting the colony refuse pile

bull Nest maintenance work construction and clearing of chambers

ndash An antrsquos decision whether to perform a task depends onbull The Phisical State of the environment

ndash If part of the nest is damaged more ants do nest maintenance work to repair it

bull Social Interactions with other ants

Communication (direct or indirect) is necessary

23418 33

bull They establish indirect communication system based on the deposition of pheromone over the path they followndash An isolated ant moves at random but when it finds a pheromone

trail there is a high probability that this ant will decide to follow the trail

ndash An ant foraging for food deposits pheromone over its route When it finds a food source it returns to the nest reinforcing its trail

ndash So other ants have greater probability to start following this trail and thereby laying more pheromone on it

ndash This process works as a positive feedback loop system because the higher the intensity of the pheromone over a trail the higher the probability of an ant start traveling through it

How can the natural ants find the shortest path

23418 34

bull Since the route B is shorter the ants on this path will complete the travel more times and thereby lay more pheromone over it

bull The pheromone concentration on trail B will increase at a higher rate than on A and soon the ants on route A will choose to follow route B

bull Since most ants will no longer travel on route A and since the pheromone is volatile trail A will start evaporating

bull Only the shortest route will remain

23418 35

Problems of ASbull Ant System tends to converge quickly

ndash This means that its exploitation of the best solution found is too high it should be exploring solution space more

ndash Pheromone evaporationupdate rule (better rule may exist)

bull Led to extensions of the ant systemndash Elitist Strategy for Ant Systems (EAS)ndash Rank based Ant Systems (AKRANK)ndash MAX-MIN Ant system (MMAS)ndash Ant Colony System (ACS)

23418 36

Part Ⅲ Particle Swarm Optimization (PSO)

1048713Firstly Proposed by Kennedy and Eberhart 1995

ldquoOnce again nature has provided us with a technique for processing information that is at once elegant and versatilerdquo

23418 37

bull In PSO a ldquoswarmrdquo is defined as an apparently disorganized collection (population) of moving individuals that tend to cluster together while each individual seems to be moving in a random direction

Bird flocking is one of the best example of PSO in nature

23418 38

Modeling bird flocking

bull The synchrony of flocking behavior is thought to be a function of birdrsquos efforts to maintain an optimal distance between themselves and their neighborsndash Birds and fish adjust their physical movement to avoid

predators seek for food and mates ndash Humans tend to adjust our beliefs and attitudes to

conform with those of our social peers Humans change in abstract multidimensional space collision-free

23418 39

Modeling bird flocking (Contrsquod)

bull Definitions ndash Flock is a group of objects that exhibit the general

class of polarized (aligned) non-colliding aggregate motion

ndash Boid is a simulated bird-like object ie it exhibits this type of behavior It can be a fish bee dinosaur etc

bull Rules for flockingndash Cohesion Each boid fly towards the centroid of its

local flock mates (that is boid in its local neighborhood)ndash Separation Each boid keep a certain distance away

from local flock mates to avoid collisionsndash Alignment Each boid align its velocity vector and keep

velocity magnitude similar with that of the local flock

23418 40

bull Imagine a birdrsquos flock in an area where there is a single food source

bull A bird donrsquot know where the food is but it knows its distance to the food

bull The best strategy is to follow the bird that is closer to the food

bull Particles save and communicate the best solution they have found

From Bird to Particle

23418 41

Features of PSO

bull Population initialized by assigning random positions and velocities potential solutions are then flown through hyperspace

bull Each particle keeps track of its ldquobestrdquo (highest fitness) position in hyperspacendash This is called ldquopBestrdquo for an individual particlendash It is called ldquogBestrdquo for the best in the populationndash It is called ldquolBestrdquo for the best in a defined neighborhood

bull At each time step each particle stochastically accelerates toward its pBest and gBest (or lBest)

23418 42

Particle Swarm Optimization Process

Step1 Initialize population in hyperspace

Step2 Evaluate fitness of individual particles

Step3 Modify velocities based on previous best and global (or neighborhood) best

Step4 Terminate on some condition

Step5 Go to step 2

23418 43

How do particles fly

bull Combination of gBest and the pBest (lBest)ndash need a compromise

bull lBest can bendash Social the particles around are always the same no matter

where they are in spacendash Geographical the particles around are those whose distance is

the shortest

bull Global PSO vs Local PSOndash the global version converges quickly to a solution but it gets

more easily stuck in local minima

23418 44

Illustrating the velocity update schema of global PSO

23418 45

PSO Related issues

bull Controlling velocities (determining the best value for Vmax)ndash Usually set maximum velocity to dynamic range of variable

bull Usually set c1 and c2 to 2

bull Inertia weight influences tradeoff between global and local explorationndash Good approach is to reduce inertia weight during run (ie from

09 to 04 over 1000 generations)

bull Swarm Size and Neighborhood Size

23418 46

Advantages of PSO

bull Adaptation operates on velocitiesndash Most other methods operate on positionsndash Effect PSO has a builtin momentumndash Particles tend to hurdle past optima ndash an advantage

since the best positions are remembered anyway

bull Simple in conceptbull Easy to implementbull Computationally efficientbull Effective on a variety of problems

23418 47

Summarybull Swarm Intelligence (SI)

ndash an artificial intelligence technique based around the study of collective behavior in decentralized self-organized systems

bull Multi-Agent Systems (MAS)ndash A system that consists of a number of agents

which interact with one-anotherndash Communication Coordination

Collaboration

23418 48

bull Ant Colony Optimization (ACO)ndash Inspired by ant colony foragingndash pheromone as heuristic information

(stigmergy)ndash Iteration between ConstructAntSolutions and

UpdatePheromones

bull Particle Swarm Optimization (PSO)ndash Inspired by bird flockingndash Heuristic information results from partnersndash Particle Velocity Update

• Collective Intelligence
• Outline
• The Computational Beauty of Nature
• Examples of Collective Behavior in Nature and Society
• Emergence
• Biological motivation Insect Societies
• Insect Societies
• Self Organization
• Stigmergy
• Flocks Herds and Schools
• Boid rules
• Slide 12
• Swarm Intelligence (Contrsquod)
• Components of SI
• Characteristics of SI
• The Web becomes a Giant Brain Some see the Web evolving into a collective brain for humankind
• What is Multi-agent Systems
• What is MAS(Contrsquod)
• Slide 19
• Communication models
• Working together
• Example mechanism Contract Net Protocol (CNP)
• Slide 23
• CNP (Contrsquod)
• Slide 25
• Attributes of Multi-agent Systems
• Applications of MAS
• Applications of MAS (Contrsquod)
• How to simulate SI for search
• Part Ⅱ Ant Colony Optimization (ACO)
• Natural Ants
• Slide 32
• How can the natural ants find the shortest path
• Slide 34
• Problems of AS
• Part Ⅲ Particle Swarm Optimization (PSO)
• Slide 37
• Modeling bird flocking
• Modeling bird flocking (Contrsquod)
• From Bird to Particle
• Features of PSO
• Particle Swarm Optimization Process
• How do particles fly
• Slide 44
• PSO Related issues
• Advantages of PSO
• Summary
• Slide 48

23418 27

Applications of MAS

Advanced Manufacturing Management Systemsminus Agents as representatives of machines users business

processes etc

Intelligent Information Search on Internetminus Some agents may show learning capabilities (learn the

preferences of their users )

Intelligent security enforcement on Internetminus Agents are representative of sensors or IDSs

Shopping Agents in Electronic Commerceminus With search price comparison and bargaining capabilities

23418 28

Applications of MAS (Contrsquod)

Multi-agent auction in E-commerce

Distributed Surveillanceminus For information search or to look for special events informing

their users of relevant news

Distributed Signal Processingminus For problem diagnosis situation assessment etc in the

network

Distributed Problem Solvingminus Collaborative design scheduling and planning

23418 29

How to simulate SI for search

Example1 Ant --gt Ant Colony Optimization (ACO)

Example2 Bird Flocking --gt Particle Swarm Optimization (PSO)

23418 30

Part Ant Colony Optimization Ⅱ(ACO)

First proposed by M Dorigo 1992

23418 31

Natural Antsbull Individual ants are simple insects with

limited memory and capable of performing simple actions

bull However an ant colony expresses a complex collective behavior providing intelligent solutions to problems such asndash carrying large itemsndash forming bridgesndash finding the shortest routes from the nest

to a food source prioritizing food sources based on their distance and ease of access

23418 32

Natural Antsbull Moreover in a colony each ant has its prescribed task but

the ants can switch tasks if the collective needs it ndash Outside the nest ants can have 4 different tasks

bull Foraging searching for and retrieving food

bull Patrolling looking for food supply

bull Midden work Sorting the colony refuse pile

bull Nest maintenance work construction and clearing of chambers

ndash An antrsquos decision whether to perform a task depends onbull The Phisical State of the environment

ndash If part of the nest is damaged more ants do nest maintenance work to repair it

bull Social Interactions with other ants

Communication (direct or indirect) is necessary

23418 33

bull They establish indirect communication system based on the deposition of pheromone over the path they followndash An isolated ant moves at random but when it finds a pheromone

trail there is a high probability that this ant will decide to follow the trail

ndash An ant foraging for food deposits pheromone over its route When it finds a food source it returns to the nest reinforcing its trail

ndash So other ants have greater probability to start following this trail and thereby laying more pheromone on it

ndash This process works as a positive feedback loop system because the higher the intensity of the pheromone over a trail the higher the probability of an ant start traveling through it

How can the natural ants find the shortest path

23418 34

bull Since the route B is shorter the ants on this path will complete the travel more times and thereby lay more pheromone over it

bull The pheromone concentration on trail B will increase at a higher rate than on A and soon the ants on route A will choose to follow route B

bull Since most ants will no longer travel on route A and since the pheromone is volatile trail A will start evaporating

bull Only the shortest route will remain

23418 35

Problems of ASbull Ant System tends to converge quickly

ndash This means that its exploitation of the best solution found is too high it should be exploring solution space more

ndash Pheromone evaporationupdate rule (better rule may exist)

bull Led to extensions of the ant systemndash Elitist Strategy for Ant Systems (EAS)ndash Rank based Ant Systems (AKRANK)ndash MAX-MIN Ant system (MMAS)ndash Ant Colony System (ACS)

23418 36

Part Ⅲ Particle Swarm Optimization (PSO)

1048713Firstly Proposed by Kennedy and Eberhart 1995

ldquoOnce again nature has provided us with a technique for processing information that is at once elegant and versatilerdquo

23418 37

bull In PSO a ldquoswarmrdquo is defined as an apparently disorganized collection (population) of moving individuals that tend to cluster together while each individual seems to be moving in a random direction

Bird flocking is one of the best example of PSO in nature

23418 38

Modeling bird flocking

bull The synchrony of flocking behavior is thought to be a function of birdrsquos efforts to maintain an optimal distance between themselves and their neighborsndash Birds and fish adjust their physical movement to avoid

predators seek for food and mates ndash Humans tend to adjust our beliefs and attitudes to

conform with those of our social peers Humans change in abstract multidimensional space collision-free

23418 39

Modeling bird flocking (Contrsquod)

bull Definitions ndash Flock is a group of objects that exhibit the general

class of polarized (aligned) non-colliding aggregate motion

ndash Boid is a simulated bird-like object ie it exhibits this type of behavior It can be a fish bee dinosaur etc

bull Rules for flockingndash Cohesion Each boid fly towards the centroid of its

local flock mates (that is boid in its local neighborhood)ndash Separation Each boid keep a certain distance away

from local flock mates to avoid collisionsndash Alignment Each boid align its velocity vector and keep

velocity magnitude similar with that of the local flock

23418 40

bull Imagine a birdrsquos flock in an area where there is a single food source

bull A bird donrsquot know where the food is but it knows its distance to the food

bull The best strategy is to follow the bird that is closer to the food

bull Particles save and communicate the best solution they have found

From Bird to Particle

23418 41

Features of PSO

bull Population initialized by assigning random positions and velocities potential solutions are then flown through hyperspace

bull Each particle keeps track of its ldquobestrdquo (highest fitness) position in hyperspacendash This is called ldquopBestrdquo for an individual particlendash It is called ldquogBestrdquo for the best in the populationndash It is called ldquolBestrdquo for the best in a defined neighborhood

bull At each time step each particle stochastically accelerates toward its pBest and gBest (or lBest)

23418 42

Particle Swarm Optimization Process

Step1 Initialize population in hyperspace

Step2 Evaluate fitness of individual particles

Step3 Modify velocities based on previous best and global (or neighborhood) best

Step4 Terminate on some condition

Step5 Go to step 2

23418 43

How do particles fly

bull Combination of gBest and the pBest (lBest)ndash need a compromise

bull lBest can bendash Social the particles around are always the same no matter

where they are in spacendash Geographical the particles around are those whose distance is

the shortest

bull Global PSO vs Local PSOndash the global version converges quickly to a solution but it gets

more easily stuck in local minima

23418 44

Illustrating the velocity update schema of global PSO

23418 45

PSO Related issues

bull Controlling velocities (determining the best value for Vmax)ndash Usually set maximum velocity to dynamic range of variable

bull Usually set c1 and c2 to 2

bull Inertia weight influences tradeoff between global and local explorationndash Good approach is to reduce inertia weight during run (ie from

09 to 04 over 1000 generations)

bull Swarm Size and Neighborhood Size

23418 46

Advantages of PSO

bull Adaptation operates on velocitiesndash Most other methods operate on positionsndash Effect PSO has a builtin momentumndash Particles tend to hurdle past optima ndash an advantage

since the best positions are remembered anyway

bull Simple in conceptbull Easy to implementbull Computationally efficientbull Effective on a variety of problems

23418 47

Summarybull Swarm Intelligence (SI)

ndash an artificial intelligence technique based around the study of collective behavior in decentralized self-organized systems

bull Multi-Agent Systems (MAS)ndash A system that consists of a number of agents

which interact with one-anotherndash Communication Coordination

Collaboration

23418 48

bull Ant Colony Optimization (ACO)ndash Inspired by ant colony foragingndash pheromone as heuristic information

(stigmergy)ndash Iteration between ConstructAntSolutions and

UpdatePheromones

bull Particle Swarm Optimization (PSO)ndash Inspired by bird flockingndash Heuristic information results from partnersndash Particle Velocity Update

• Collective Intelligence
• Outline
• The Computational Beauty of Nature
• Examples of Collective Behavior in Nature and Society
• Emergence
• Biological motivation Insect Societies
• Insect Societies
• Self Organization
• Stigmergy
• Flocks Herds and Schools
• Boid rules
• Slide 12
• Swarm Intelligence (Contrsquod)
• Components of SI
• Characteristics of SI
• The Web becomes a Giant Brain Some see the Web evolving into a collective brain for humankind
• What is Multi-agent Systems
• What is MAS(Contrsquod)
• Slide 19
• Communication models
• Working together
• Example mechanism Contract Net Protocol (CNP)
• Slide 23
• CNP (Contrsquod)
• Slide 25
• Attributes of Multi-agent Systems
• Applications of MAS
• Applications of MAS (Contrsquod)
• How to simulate SI for search
• Part Ⅱ Ant Colony Optimization (ACO)
• Natural Ants
• Slide 32
• How can the natural ants find the shortest path
• Slide 34
• Problems of AS
• Part Ⅲ Particle Swarm Optimization (PSO)
• Slide 37
• Modeling bird flocking
• Modeling bird flocking (Contrsquod)
• From Bird to Particle
• Features of PSO
• Particle Swarm Optimization Process
• How do particles fly
• Slide 44
• PSO Related issues
• Advantages of PSO
• Summary
• Slide 48

23418 28

Applications of MAS (Contrsquod)

Multi-agent auction in E-commerce

Distributed Surveillanceminus For information search or to look for special events informing

their users of relevant news

Distributed Signal Processingminus For problem diagnosis situation assessment etc in the

network

Distributed Problem Solvingminus Collaborative design scheduling and planning

23418 29

How to simulate SI for search

Example1 Ant --gt Ant Colony Optimization (ACO)

Example2 Bird Flocking --gt Particle Swarm Optimization (PSO)

23418 30

Part Ant Colony Optimization Ⅱ(ACO)

First proposed by M Dorigo 1992

23418 31

Natural Antsbull Individual ants are simple insects with

limited memory and capable of performing simple actions

bull However an ant colony expresses a complex collective behavior providing intelligent solutions to problems such asndash carrying large itemsndash forming bridgesndash finding the shortest routes from the nest

to a food source prioritizing food sources based on their distance and ease of access

23418 32

Natural Antsbull Moreover in a colony each ant has its prescribed task but

the ants can switch tasks if the collective needs it ndash Outside the nest ants can have 4 different tasks

bull Foraging searching for and retrieving food

bull Patrolling looking for food supply

bull Midden work Sorting the colony refuse pile

bull Nest maintenance work construction and clearing of chambers

ndash An antrsquos decision whether to perform a task depends onbull The Phisical State of the environment

ndash If part of the nest is damaged more ants do nest maintenance work to repair it

bull Social Interactions with other ants

Communication (direct or indirect) is necessary

23418 33

bull They establish indirect communication system based on the deposition of pheromone over the path they followndash An isolated ant moves at random but when it finds a pheromone

trail there is a high probability that this ant will decide to follow the trail

ndash An ant foraging for food deposits pheromone over its route When it finds a food source it returns to the nest reinforcing its trail

ndash So other ants have greater probability to start following this trail and thereby laying more pheromone on it

ndash This process works as a positive feedback loop system because the higher the intensity of the pheromone over a trail the higher the probability of an ant start traveling through it

How can the natural ants find the shortest path

23418 34

bull Since the route B is shorter the ants on this path will complete the travel more times and thereby lay more pheromone over it

bull The pheromone concentration on trail B will increase at a higher rate than on A and soon the ants on route A will choose to follow route B

bull Since most ants will no longer travel on route A and since the pheromone is volatile trail A will start evaporating

bull Only the shortest route will remain

23418 35

Problems of ASbull Ant System tends to converge quickly

ndash This means that its exploitation of the best solution found is too high it should be exploring solution space more

ndash Pheromone evaporationupdate rule (better rule may exist)

bull Led to extensions of the ant systemndash Elitist Strategy for Ant Systems (EAS)ndash Rank based Ant Systems (AKRANK)ndash MAX-MIN Ant system (MMAS)ndash Ant Colony System (ACS)

23418 36

Part Ⅲ Particle Swarm Optimization (PSO)

1048713Firstly Proposed by Kennedy and Eberhart 1995

ldquoOnce again nature has provided us with a technique for processing information that is at once elegant and versatilerdquo

23418 37

bull In PSO a ldquoswarmrdquo is defined as an apparently disorganized collection (population) of moving individuals that tend to cluster together while each individual seems to be moving in a random direction

Bird flocking is one of the best example of PSO in nature

23418 38

Modeling bird flocking

bull The synchrony of flocking behavior is thought to be a function of birdrsquos efforts to maintain an optimal distance between themselves and their neighborsndash Birds and fish adjust their physical movement to avoid

predators seek for food and mates ndash Humans tend to adjust our beliefs and attitudes to

conform with those of our social peers Humans change in abstract multidimensional space collision-free

23418 39

Modeling bird flocking (Contrsquod)

bull Definitions ndash Flock is a group of objects that exhibit the general

class of polarized (aligned) non-colliding aggregate motion

ndash Boid is a simulated bird-like object ie it exhibits this type of behavior It can be a fish bee dinosaur etc

bull Rules for flockingndash Cohesion Each boid fly towards the centroid of its

local flock mates (that is boid in its local neighborhood)ndash Separation Each boid keep a certain distance away

from local flock mates to avoid collisionsndash Alignment Each boid align its velocity vector and keep

velocity magnitude similar with that of the local flock

23418 40

bull Imagine a birdrsquos flock in an area where there is a single food source

bull A bird donrsquot know where the food is but it knows its distance to the food

bull The best strategy is to follow the bird that is closer to the food

bull Particles save and communicate the best solution they have found

From Bird to Particle

23418 41

Features of PSO

bull Population initialized by assigning random positions and velocities potential solutions are then flown through hyperspace

bull Each particle keeps track of its ldquobestrdquo (highest fitness) position in hyperspacendash This is called ldquopBestrdquo for an individual particlendash It is called ldquogBestrdquo for the best in the populationndash It is called ldquolBestrdquo for the best in a defined neighborhood

bull At each time step each particle stochastically accelerates toward its pBest and gBest (or lBest)

23418 42

Particle Swarm Optimization Process

Step1 Initialize population in hyperspace

Step2 Evaluate fitness of individual particles

Step3 Modify velocities based on previous best and global (or neighborhood) best

Step4 Terminate on some condition

Step5 Go to step 2

23418 43

How do particles fly

bull Combination of gBest and the pBest (lBest)ndash need a compromise

bull lBest can bendash Social the particles around are always the same no matter

where they are in spacendash Geographical the particles around are those whose distance is

the shortest

bull Global PSO vs Local PSOndash the global version converges quickly to a solution but it gets

more easily stuck in local minima

23418 44

Illustrating the velocity update schema of global PSO

23418 45

PSO Related issues

bull Controlling velocities (determining the best value for Vmax)ndash Usually set maximum velocity to dynamic range of variable

bull Usually set c1 and c2 to 2

bull Inertia weight influences tradeoff between global and local explorationndash Good approach is to reduce inertia weight during run (ie from

09 to 04 over 1000 generations)

bull Swarm Size and Neighborhood Size

23418 46

Advantages of PSO

bull Adaptation operates on velocitiesndash Most other methods operate on positionsndash Effect PSO has a builtin momentumndash Particles tend to hurdle past optima ndash an advantage

since the best positions are remembered anyway

bull Simple in conceptbull Easy to implementbull Computationally efficientbull Effective on a variety of problems

23418 47

Summarybull Swarm Intelligence (SI)

ndash an artificial intelligence technique based around the study of collective behavior in decentralized self-organized systems

bull Multi-Agent Systems (MAS)ndash A system that consists of a number of agents

which interact with one-anotherndash Communication Coordination

Collaboration

23418 48

bull Ant Colony Optimization (ACO)ndash Inspired by ant colony foragingndash pheromone as heuristic information

(stigmergy)ndash Iteration between ConstructAntSolutions and

UpdatePheromones

bull Particle Swarm Optimization (PSO)ndash Inspired by bird flockingndash Heuristic information results from partnersndash Particle Velocity Update

• Collective Intelligence
• Outline
• The Computational Beauty of Nature
• Examples of Collective Behavior in Nature and Society
• Emergence
• Biological motivation Insect Societies
• Insect Societies
• Self Organization
• Stigmergy
• Flocks Herds and Schools
• Boid rules
• Slide 12
• Swarm Intelligence (Contrsquod)
• Components of SI
• Characteristics of SI
• The Web becomes a Giant Brain Some see the Web evolving into a collective brain for humankind
• What is Multi-agent Systems
• What is MAS(Contrsquod)
• Slide 19
• Communication models
• Working together
• Example mechanism Contract Net Protocol (CNP)
• Slide 23
• CNP (Contrsquod)
• Slide 25
• Attributes of Multi-agent Systems
• Applications of MAS
• Applications of MAS (Contrsquod)
• How to simulate SI for search
• Part Ⅱ Ant Colony Optimization (ACO)
• Natural Ants
• Slide 32
• How can the natural ants find the shortest path
• Slide 34
• Problems of AS
• Part Ⅲ Particle Swarm Optimization (PSO)
• Slide 37
• Modeling bird flocking
• Modeling bird flocking (Contrsquod)
• From Bird to Particle
• Features of PSO
• Particle Swarm Optimization Process
• How do particles fly
• Slide 44
• PSO Related issues
• Advantages of PSO
• Summary
• Slide 48

23418 29

How to simulate SI for search

Example1 Ant --gt Ant Colony Optimization (ACO)

Example2 Bird Flocking --gt Particle Swarm Optimization (PSO)

23418 30

Part Ant Colony Optimization Ⅱ(ACO)

First proposed by M Dorigo 1992

23418 31

Natural Antsbull Individual ants are simple insects with

limited memory and capable of performing simple actions

bull However an ant colony expresses a complex collective behavior providing intelligent solutions to problems such asndash carrying large itemsndash forming bridgesndash finding the shortest routes from the nest

to a food source prioritizing food sources based on their distance and ease of access

23418 32

Natural Antsbull Moreover in a colony each ant has its prescribed task but

the ants can switch tasks if the collective needs it ndash Outside the nest ants can have 4 different tasks

bull Foraging searching for and retrieving food

bull Patrolling looking for food supply

bull Midden work Sorting the colony refuse pile

bull Nest maintenance work construction and clearing of chambers

ndash An antrsquos decision whether to perform a task depends onbull The Phisical State of the environment

ndash If part of the nest is damaged more ants do nest maintenance work to repair it

bull Social Interactions with other ants

Communication (direct or indirect) is necessary

23418 33

bull They establish indirect communication system based on the deposition of pheromone over the path they followndash An isolated ant moves at random but when it finds a pheromone

trail there is a high probability that this ant will decide to follow the trail

ndash An ant foraging for food deposits pheromone over its route When it finds a food source it returns to the nest reinforcing its trail

ndash So other ants have greater probability to start following this trail and thereby laying more pheromone on it

ndash This process works as a positive feedback loop system because the higher the intensity of the pheromone over a trail the higher the probability of an ant start traveling through it

How can the natural ants find the shortest path

23418 34

bull Since the route B is shorter the ants on this path will complete the travel more times and thereby lay more pheromone over it

bull The pheromone concentration on trail B will increase at a higher rate than on A and soon the ants on route A will choose to follow route B

bull Since most ants will no longer travel on route A and since the pheromone is volatile trail A will start evaporating

bull Only the shortest route will remain

23418 35

Problems of ASbull Ant System tends to converge quickly

ndash This means that its exploitation of the best solution found is too high it should be exploring solution space more

ndash Pheromone evaporationupdate rule (better rule may exist)

bull Led to extensions of the ant systemndash Elitist Strategy for Ant Systems (EAS)ndash Rank based Ant Systems (AKRANK)ndash MAX-MIN Ant system (MMAS)ndash Ant Colony System (ACS)

23418 36

Part Ⅲ Particle Swarm Optimization (PSO)

1048713Firstly Proposed by Kennedy and Eberhart 1995

ldquoOnce again nature has provided us with a technique for processing information that is at once elegant and versatilerdquo

23418 37

bull In PSO a ldquoswarmrdquo is defined as an apparently disorganized collection (population) of moving individuals that tend to cluster together while each individual seems to be moving in a random direction

Bird flocking is one of the best example of PSO in nature

23418 38

Modeling bird flocking

bull The synchrony of flocking behavior is thought to be a function of birdrsquos efforts to maintain an optimal distance between themselves and their neighborsndash Birds and fish adjust their physical movement to avoid

predators seek for food and mates ndash Humans tend to adjust our beliefs and attitudes to

conform with those of our social peers Humans change in abstract multidimensional space collision-free

23418 39

Modeling bird flocking (Contrsquod)

bull Definitions ndash Flock is a group of objects that exhibit the general

class of polarized (aligned) non-colliding aggregate motion

ndash Boid is a simulated bird-like object ie it exhibits this type of behavior It can be a fish bee dinosaur etc

bull Rules for flockingndash Cohesion Each boid fly towards the centroid of its

local flock mates (that is boid in its local neighborhood)ndash Separation Each boid keep a certain distance away

from local flock mates to avoid collisionsndash Alignment Each boid align its velocity vector and keep

velocity magnitude similar with that of the local flock

23418 40

bull Imagine a birdrsquos flock in an area where there is a single food source

bull A bird donrsquot know where the food is but it knows its distance to the food

bull The best strategy is to follow the bird that is closer to the food

bull Particles save and communicate the best solution they have found

From Bird to Particle

23418 41

Features of PSO

bull Population initialized by assigning random positions and velocities potential solutions are then flown through hyperspace

bull Each particle keeps track of its ldquobestrdquo (highest fitness) position in hyperspacendash This is called ldquopBestrdquo for an individual particlendash It is called ldquogBestrdquo for the best in the populationndash It is called ldquolBestrdquo for the best in a defined neighborhood

bull At each time step each particle stochastically accelerates toward its pBest and gBest (or lBest)

23418 42

Particle Swarm Optimization Process

Step1 Initialize population in hyperspace

Step2 Evaluate fitness of individual particles

Step3 Modify velocities based on previous best and global (or neighborhood) best

Step4 Terminate on some condition

Step5 Go to step 2

23418 43

How do particles fly

bull Combination of gBest and the pBest (lBest)ndash need a compromise

bull lBest can bendash Social the particles around are always the same no matter

where they are in spacendash Geographical the particles around are those whose distance is

the shortest

bull Global PSO vs Local PSOndash the global version converges quickly to a solution but it gets

more easily stuck in local minima

23418 44

Illustrating the velocity update schema of global PSO

23418 45

PSO Related issues

bull Controlling velocities (determining the best value for Vmax)ndash Usually set maximum velocity to dynamic range of variable

bull Usually set c1 and c2 to 2

bull Inertia weight influences tradeoff between global and local explorationndash Good approach is to reduce inertia weight during run (ie from

09 to 04 over 1000 generations)

bull Swarm Size and Neighborhood Size

23418 46

Advantages of PSO

bull Adaptation operates on velocitiesndash Most other methods operate on positionsndash Effect PSO has a builtin momentumndash Particles tend to hurdle past optima ndash an advantage

since the best positions are remembered anyway

bull Simple in conceptbull Easy to implementbull Computationally efficientbull Effective on a variety of problems

23418 47

Summarybull Swarm Intelligence (SI)

ndash an artificial intelligence technique based around the study of collective behavior in decentralized self-organized systems

bull Multi-Agent Systems (MAS)ndash A system that consists of a number of agents

which interact with one-anotherndash Communication Coordination

Collaboration

23418 48

bull Ant Colony Optimization (ACO)ndash Inspired by ant colony foragingndash pheromone as heuristic information

(stigmergy)ndash Iteration between ConstructAntSolutions and

UpdatePheromones

bull Particle Swarm Optimization (PSO)ndash Inspired by bird flockingndash Heuristic information results from partnersndash Particle Velocity Update

• Collective Intelligence
• Outline
• The Computational Beauty of Nature
• Examples of Collective Behavior in Nature and Society
• Emergence
• Biological motivation Insect Societies
• Insect Societies
• Self Organization
• Stigmergy
• Flocks Herds and Schools
• Boid rules
• Slide 12
• Swarm Intelligence (Contrsquod)
• Components of SI
• Characteristics of SI
• The Web becomes a Giant Brain Some see the Web evolving into a collective brain for humankind
• What is Multi-agent Systems
• What is MAS(Contrsquod)
• Slide 19
• Communication models
• Working together
• Example mechanism Contract Net Protocol (CNP)
• Slide 23
• CNP (Contrsquod)
• Slide 25
• Attributes of Multi-agent Systems
• Applications of MAS
• Applications of MAS (Contrsquod)
• How to simulate SI for search
• Part Ⅱ Ant Colony Optimization (ACO)
• Natural Ants
• Slide 32
• How can the natural ants find the shortest path
• Slide 34
• Problems of AS
• Part Ⅲ Particle Swarm Optimization (PSO)
• Slide 37
• Modeling bird flocking
• Modeling bird flocking (Contrsquod)
• From Bird to Particle
• Features of PSO
• Particle Swarm Optimization Process
• How do particles fly
• Slide 44
• PSO Related issues
• Advantages of PSO
• Summary
• Slide 48

23418 30

Part Ant Colony Optimization Ⅱ(ACO)

First proposed by M Dorigo 1992

23418 31

Natural Antsbull Individual ants are simple insects with

limited memory and capable of performing simple actions

bull However an ant colony expresses a complex collective behavior providing intelligent solutions to problems such asndash carrying large itemsndash forming bridgesndash finding the shortest routes from the nest

to a food source prioritizing food sources based on their distance and ease of access

23418 32

Natural Antsbull Moreover in a colony each ant has its prescribed task but

the ants can switch tasks if the collective needs it ndash Outside the nest ants can have 4 different tasks

bull Foraging searching for and retrieving food

bull Patrolling looking for food supply

bull Midden work Sorting the colony refuse pile

bull Nest maintenance work construction and clearing of chambers

ndash An antrsquos decision whether to perform a task depends onbull The Phisical State of the environment

ndash If part of the nest is damaged more ants do nest maintenance work to repair it

bull Social Interactions with other ants

Communication (direct or indirect) is necessary

23418 33

bull They establish indirect communication system based on the deposition of pheromone over the path they followndash An isolated ant moves at random but when it finds a pheromone

trail there is a high probability that this ant will decide to follow the trail

ndash An ant foraging for food deposits pheromone over its route When it finds a food source it returns to the nest reinforcing its trail

ndash So other ants have greater probability to start following this trail and thereby laying more pheromone on it

ndash This process works as a positive feedback loop system because the higher the intensity of the pheromone over a trail the higher the probability of an ant start traveling through it

How can the natural ants find the shortest path

23418 34

bull Since the route B is shorter the ants on this path will complete the travel more times and thereby lay more pheromone over it

bull The pheromone concentration on trail B will increase at a higher rate than on A and soon the ants on route A will choose to follow route B

bull Since most ants will no longer travel on route A and since the pheromone is volatile trail A will start evaporating

bull Only the shortest route will remain

23418 35

Problems of ASbull Ant System tends to converge quickly

ndash This means that its exploitation of the best solution found is too high it should be exploring solution space more

ndash Pheromone evaporationupdate rule (better rule may exist)

bull Led to extensions of the ant systemndash Elitist Strategy for Ant Systems (EAS)ndash Rank based Ant Systems (AKRANK)ndash MAX-MIN Ant system (MMAS)ndash Ant Colony System (ACS)

23418 36

Part Ⅲ Particle Swarm Optimization (PSO)

1048713Firstly Proposed by Kennedy and Eberhart 1995

ldquoOnce again nature has provided us with a technique for processing information that is at once elegant and versatilerdquo

23418 37

bull In PSO a ldquoswarmrdquo is defined as an apparently disorganized collection (population) of moving individuals that tend to cluster together while each individual seems to be moving in a random direction

Bird flocking is one of the best example of PSO in nature

23418 38

Modeling bird flocking

bull The synchrony of flocking behavior is thought to be a function of birdrsquos efforts to maintain an optimal distance between themselves and their neighborsndash Birds and fish adjust their physical movement to avoid

predators seek for food and mates ndash Humans tend to adjust our beliefs and attitudes to

conform with those of our social peers Humans change in abstract multidimensional space collision-free

23418 39

Modeling bird flocking (Contrsquod)

bull Definitions ndash Flock is a group of objects that exhibit the general

class of polarized (aligned) non-colliding aggregate motion

ndash Boid is a simulated bird-like object ie it exhibits this type of behavior It can be a fish bee dinosaur etc

bull Rules for flockingndash Cohesion Each boid fly towards the centroid of its

local flock mates (that is boid in its local neighborhood)ndash Separation Each boid keep a certain distance away

from local flock mates to avoid collisionsndash Alignment Each boid align its velocity vector and keep

velocity magnitude similar with that of the local flock

23418 40

bull Imagine a birdrsquos flock in an area where there is a single food source

bull A bird donrsquot know where the food is but it knows its distance to the food

bull The best strategy is to follow the bird that is closer to the food

bull Particles save and communicate the best solution they have found

From Bird to Particle

23418 41

Features of PSO

bull Population initialized by assigning random positions and velocities potential solutions are then flown through hyperspace

bull Each particle keeps track of its ldquobestrdquo (highest fitness) position in hyperspacendash This is called ldquopBestrdquo for an individual particlendash It is called ldquogBestrdquo for the best in the populationndash It is called ldquolBestrdquo for the best in a defined neighborhood

bull At each time step each particle stochastically accelerates toward its pBest and gBest (or lBest)

23418 42

Particle Swarm Optimization Process

Step1 Initialize population in hyperspace

Step2 Evaluate fitness of individual particles

Step3 Modify velocities based on previous best and global (or neighborhood) best

Step4 Terminate on some condition

Step5 Go to step 2

23418 43

How do particles fly

bull Combination of gBest and the pBest (lBest)ndash need a compromise

bull lBest can bendash Social the particles around are always the same no matter

where they are in spacendash Geographical the particles around are those whose distance is

the shortest

bull Global PSO vs Local PSOndash the global version converges quickly to a solution but it gets

more easily stuck in local minima

23418 44

Illustrating the velocity update schema of global PSO

23418 45

PSO Related issues

bull Controlling velocities (determining the best value for Vmax)ndash Usually set maximum velocity to dynamic range of variable

bull Usually set c1 and c2 to 2

bull Inertia weight influences tradeoff between global and local explorationndash Good approach is to reduce inertia weight during run (ie from

09 to 04 over 1000 generations)

bull Swarm Size and Neighborhood Size

23418 46

Advantages of PSO

bull Adaptation operates on velocitiesndash Most other methods operate on positionsndash Effect PSO has a builtin momentumndash Particles tend to hurdle past optima ndash an advantage

since the best positions are remembered anyway

bull Simple in conceptbull Easy to implementbull Computationally efficientbull Effective on a variety of problems

23418 47

Summarybull Swarm Intelligence (SI)

ndash an artificial intelligence technique based around the study of collective behavior in decentralized self-organized systems

bull Multi-Agent Systems (MAS)ndash A system that consists of a number of agents

which interact with one-anotherndash Communication Coordination

Collaboration

23418 48

bull Ant Colony Optimization (ACO)ndash Inspired by ant colony foragingndash pheromone as heuristic information

(stigmergy)ndash Iteration between ConstructAntSolutions and

UpdatePheromones

bull Particle Swarm Optimization (PSO)ndash Inspired by bird flockingndash Heuristic information results from partnersndash Particle Velocity Update

• Collective Intelligence
• Outline
• The Computational Beauty of Nature
• Examples of Collective Behavior in Nature and Society
• Emergence
• Biological motivation Insect Societies
• Insect Societies
• Self Organization
• Stigmergy
• Flocks Herds and Schools
• Boid rules
• Slide 12
• Swarm Intelligence (Contrsquod)
• Components of SI
• Characteristics of SI
• The Web becomes a Giant Brain Some see the Web evolving into a collective brain for humankind
• What is Multi-agent Systems
• What is MAS(Contrsquod)
• Slide 19
• Communication models
• Working together
• Example mechanism Contract Net Protocol (CNP)
• Slide 23
• CNP (Contrsquod)
• Slide 25
• Attributes of Multi-agent Systems
• Applications of MAS
• Applications of MAS (Contrsquod)
• How to simulate SI for search
• Part Ⅱ Ant Colony Optimization (ACO)
• Natural Ants
• Slide 32
• How can the natural ants find the shortest path
• Slide 34
• Problems of AS
• Part Ⅲ Particle Swarm Optimization (PSO)
• Slide 37
• Modeling bird flocking
• Modeling bird flocking (Contrsquod)
• From Bird to Particle
• Features of PSO
• Particle Swarm Optimization Process
• How do particles fly
• Slide 44
• PSO Related issues
• Advantages of PSO
• Summary
• Slide 48

23418 31

Natural Antsbull Individual ants are simple insects with

limited memory and capable of performing simple actions

bull However an ant colony expresses a complex collective behavior providing intelligent solutions to problems such asndash carrying large itemsndash forming bridgesndash finding the shortest routes from the nest

to a food source prioritizing food sources based on their distance and ease of access

23418 32

Natural Antsbull Moreover in a colony each ant has its prescribed task but

the ants can switch tasks if the collective needs it ndash Outside the nest ants can have 4 different tasks

bull Foraging searching for and retrieving food

bull Patrolling looking for food supply

bull Midden work Sorting the colony refuse pile

bull Nest maintenance work construction and clearing of chambers

ndash An antrsquos decision whether to perform a task depends onbull The Phisical State of the environment

ndash If part of the nest is damaged more ants do nest maintenance work to repair it

bull Social Interactions with other ants

Communication (direct or indirect) is necessary

23418 33

bull They establish indirect communication system based on the deposition of pheromone over the path they followndash An isolated ant moves at random but when it finds a pheromone

trail there is a high probability that this ant will decide to follow the trail

ndash An ant foraging for food deposits pheromone over its route When it finds a food source it returns to the nest reinforcing its trail

ndash So other ants have greater probability to start following this trail and thereby laying more pheromone on it

ndash This process works as a positive feedback loop system because the higher the intensity of the pheromone over a trail the higher the probability of an ant start traveling through it

How can the natural ants find the shortest path

23418 34

bull Since the route B is shorter the ants on this path will complete the travel more times and thereby lay more pheromone over it

bull The pheromone concentration on trail B will increase at a higher rate than on A and soon the ants on route A will choose to follow route B

bull Since most ants will no longer travel on route A and since the pheromone is volatile trail A will start evaporating

bull Only the shortest route will remain

23418 35

Problems of ASbull Ant System tends to converge quickly

ndash This means that its exploitation of the best solution found is too high it should be exploring solution space more

ndash Pheromone evaporationupdate rule (better rule may exist)

bull Led to extensions of the ant systemndash Elitist Strategy for Ant Systems (EAS)ndash Rank based Ant Systems (AKRANK)ndash MAX-MIN Ant system (MMAS)ndash Ant Colony System (ACS)

23418 36

Part Ⅲ Particle Swarm Optimization (PSO)

1048713Firstly Proposed by Kennedy and Eberhart 1995

ldquoOnce again nature has provided us with a technique for processing information that is at once elegant and versatilerdquo

23418 37

bull In PSO a ldquoswarmrdquo is defined as an apparently disorganized collection (population) of moving individuals that tend to cluster together while each individual seems to be moving in a random direction

Bird flocking is one of the best example of PSO in nature

23418 38

Modeling bird flocking

bull The synchrony of flocking behavior is thought to be a function of birdrsquos efforts to maintain an optimal distance between themselves and their neighborsndash Birds and fish adjust their physical movement to avoid

predators seek for food and mates ndash Humans tend to adjust our beliefs and attitudes to

conform with those of our social peers Humans change in abstract multidimensional space collision-free

23418 39

Modeling bird flocking (Contrsquod)

bull Definitions ndash Flock is a group of objects that exhibit the general

class of polarized (aligned) non-colliding aggregate motion

ndash Boid is a simulated bird-like object ie it exhibits this type of behavior It can be a fish bee dinosaur etc

bull Rules for flockingndash Cohesion Each boid fly towards the centroid of its

local flock mates (that is boid in its local neighborhood)ndash Separation Each boid keep a certain distance away

from local flock mates to avoid collisionsndash Alignment Each boid align its velocity vector and keep

velocity magnitude similar with that of the local flock

23418 40

bull Imagine a birdrsquos flock in an area where there is a single food source

bull A bird donrsquot know where the food is but it knows its distance to the food

bull The best strategy is to follow the bird that is closer to the food

bull Particles save and communicate the best solution they have found

From Bird to Particle

23418 41

Features of PSO

bull Population initialized by assigning random positions and velocities potential solutions are then flown through hyperspace

bull Each particle keeps track of its ldquobestrdquo (highest fitness) position in hyperspacendash This is called ldquopBestrdquo for an individual particlendash It is called ldquogBestrdquo for the best in the populationndash It is called ldquolBestrdquo for the best in a defined neighborhood

bull At each time step each particle stochastically accelerates toward its pBest and gBest (or lBest)

23418 42

Particle Swarm Optimization Process

Step1 Initialize population in hyperspace

Step2 Evaluate fitness of individual particles

Step3 Modify velocities based on previous best and global (or neighborhood) best

Step4 Terminate on some condition

Step5 Go to step 2

23418 43

How do particles fly

bull Combination of gBest and the pBest (lBest)ndash need a compromise

bull lBest can bendash Social the particles around are always the same no matter

where they are in spacendash Geographical the particles around are those whose distance is

the shortest

bull Global PSO vs Local PSOndash the global version converges quickly to a solution but it gets

more easily stuck in local minima

23418 44

Illustrating the velocity update schema of global PSO

23418 45

PSO Related issues

bull Controlling velocities (determining the best value for Vmax)ndash Usually set maximum velocity to dynamic range of variable

bull Usually set c1 and c2 to 2

bull Inertia weight influences tradeoff between global and local explorationndash Good approach is to reduce inertia weight during run (ie from

09 to 04 over 1000 generations)

bull Swarm Size and Neighborhood Size

23418 46

Advantages of PSO

bull Adaptation operates on velocitiesndash Most other methods operate on positionsndash Effect PSO has a builtin momentumndash Particles tend to hurdle past optima ndash an advantage

since the best positions are remembered anyway

bull Simple in conceptbull Easy to implementbull Computationally efficientbull Effective on a variety of problems

23418 47

Summarybull Swarm Intelligence (SI)

ndash an artificial intelligence technique based around the study of collective behavior in decentralized self-organized systems

bull Multi-Agent Systems (MAS)ndash A system that consists of a number of agents

which interact with one-anotherndash Communication Coordination

Collaboration

23418 48

bull Ant Colony Optimization (ACO)ndash Inspired by ant colony foragingndash pheromone as heuristic information

(stigmergy)ndash Iteration between ConstructAntSolutions and

UpdatePheromones

bull Particle Swarm Optimization (PSO)ndash Inspired by bird flockingndash Heuristic information results from partnersndash Particle Velocity Update

• Collective Intelligence
• Outline
• The Computational Beauty of Nature
• Examples of Collective Behavior in Nature and Society
• Emergence
• Biological motivation Insect Societies
• Insect Societies
• Self Organization
• Stigmergy
• Flocks Herds and Schools
• Boid rules
• Slide 12
• Swarm Intelligence (Contrsquod)
• Components of SI
• Characteristics of SI
• The Web becomes a Giant Brain Some see the Web evolving into a collective brain for humankind
• What is Multi-agent Systems
• What is MAS(Contrsquod)
• Slide 19
• Communication models
• Working together
• Example mechanism Contract Net Protocol (CNP)
• Slide 23
• CNP (Contrsquod)
• Slide 25
• Attributes of Multi-agent Systems
• Applications of MAS
• Applications of MAS (Contrsquod)
• How to simulate SI for search
• Part Ⅱ Ant Colony Optimization (ACO)
• Natural Ants
• Slide 32
• How can the natural ants find the shortest path
• Slide 34
• Problems of AS
• Part Ⅲ Particle Swarm Optimization (PSO)
• Slide 37
• Modeling bird flocking
• Modeling bird flocking (Contrsquod)
• From Bird to Particle
• Features of PSO
• Particle Swarm Optimization Process
• How do particles fly
• Slide 44
• PSO Related issues
• Advantages of PSO
• Summary
• Slide 48

23418 32

Natural Antsbull Moreover in a colony each ant has its prescribed task but

the ants can switch tasks if the collective needs it ndash Outside the nest ants can have 4 different tasks

bull Foraging searching for and retrieving food

bull Patrolling looking for food supply

bull Midden work Sorting the colony refuse pile

bull Nest maintenance work construction and clearing of chambers

ndash An antrsquos decision whether to perform a task depends onbull The Phisical State of the environment

ndash If part of the nest is damaged more ants do nest maintenance work to repair it

bull Social Interactions with other ants

Communication (direct or indirect) is necessary

23418 33

bull They establish indirect communication system based on the deposition of pheromone over the path they followndash An isolated ant moves at random but when it finds a pheromone

trail there is a high probability that this ant will decide to follow the trail

ndash An ant foraging for food deposits pheromone over its route When it finds a food source it returns to the nest reinforcing its trail

ndash So other ants have greater probability to start following this trail and thereby laying more pheromone on it

ndash This process works as a positive feedback loop system because the higher the intensity of the pheromone over a trail the higher the probability of an ant start traveling through it

How can the natural ants find the shortest path

23418 34

bull Since the route B is shorter the ants on this path will complete the travel more times and thereby lay more pheromone over it

bull The pheromone concentration on trail B will increase at a higher rate than on A and soon the ants on route A will choose to follow route B

bull Since most ants will no longer travel on route A and since the pheromone is volatile trail A will start evaporating

bull Only the shortest route will remain

23418 35

Problems of ASbull Ant System tends to converge quickly

ndash This means that its exploitation of the best solution found is too high it should be exploring solution space more

ndash Pheromone evaporationupdate rule (better rule may exist)

bull Led to extensions of the ant systemndash Elitist Strategy for Ant Systems (EAS)ndash Rank based Ant Systems (AKRANK)ndash MAX-MIN Ant system (MMAS)ndash Ant Colony System (ACS)

23418 36

Part Ⅲ Particle Swarm Optimization (PSO)

1048713Firstly Proposed by Kennedy and Eberhart 1995

ldquoOnce again nature has provided us with a technique for processing information that is at once elegant and versatilerdquo

23418 37

bull In PSO a ldquoswarmrdquo is defined as an apparently disorganized collection (population) of moving individuals that tend to cluster together while each individual seems to be moving in a random direction

Bird flocking is one of the best example of PSO in nature

23418 38

Modeling bird flocking

bull The synchrony of flocking behavior is thought to be a function of birdrsquos efforts to maintain an optimal distance between themselves and their neighborsndash Birds and fish adjust their physical movement to avoid

predators seek for food and mates ndash Humans tend to adjust our beliefs and attitudes to

conform with those of our social peers Humans change in abstract multidimensional space collision-free

23418 39

Modeling bird flocking (Contrsquod)

bull Definitions ndash Flock is a group of objects that exhibit the general

class of polarized (aligned) non-colliding aggregate motion

ndash Boid is a simulated bird-like object ie it exhibits this type of behavior It can be a fish bee dinosaur etc

bull Rules for flockingndash Cohesion Each boid fly towards the centroid of its

local flock mates (that is boid in its local neighborhood)ndash Separation Each boid keep a certain distance away

from local flock mates to avoid collisionsndash Alignment Each boid align its velocity vector and keep

velocity magnitude similar with that of the local flock

23418 40

bull Imagine a birdrsquos flock in an area where there is a single food source

bull A bird donrsquot know where the food is but it knows its distance to the food

bull The best strategy is to follow the bird that is closer to the food

bull Particles save and communicate the best solution they have found

From Bird to Particle

23418 41

Features of PSO

bull Population initialized by assigning random positions and velocities potential solutions are then flown through hyperspace

bull Each particle keeps track of its ldquobestrdquo (highest fitness) position in hyperspacendash This is called ldquopBestrdquo for an individual particlendash It is called ldquogBestrdquo for the best in the populationndash It is called ldquolBestrdquo for the best in a defined neighborhood

bull At each time step each particle stochastically accelerates toward its pBest and gBest (or lBest)

23418 42

Particle Swarm Optimization Process

Step1 Initialize population in hyperspace

Step2 Evaluate fitness of individual particles

Step3 Modify velocities based on previous best and global (or neighborhood) best

Step4 Terminate on some condition

Step5 Go to step 2

23418 43

How do particles fly

bull Combination of gBest and the pBest (lBest)ndash need a compromise

bull lBest can bendash Social the particles around are always the same no matter

where they are in spacendash Geographical the particles around are those whose distance is

the shortest

bull Global PSO vs Local PSOndash the global version converges quickly to a solution but it gets

more easily stuck in local minima

23418 44

Illustrating the velocity update schema of global PSO

23418 45

PSO Related issues

bull Controlling velocities (determining the best value for Vmax)ndash Usually set maximum velocity to dynamic range of variable

bull Usually set c1 and c2 to 2

bull Inertia weight influences tradeoff between global and local explorationndash Good approach is to reduce inertia weight during run (ie from

09 to 04 over 1000 generations)

bull Swarm Size and Neighborhood Size

23418 46

Advantages of PSO

bull Adaptation operates on velocitiesndash Most other methods operate on positionsndash Effect PSO has a builtin momentumndash Particles tend to hurdle past optima ndash an advantage

since the best positions are remembered anyway

bull Simple in conceptbull Easy to implementbull Computationally efficientbull Effective on a variety of problems

23418 47

Summarybull Swarm Intelligence (SI)

ndash an artificial intelligence technique based around the study of collective behavior in decentralized self-organized systems

bull Multi-Agent Systems (MAS)ndash A system that consists of a number of agents

which interact with one-anotherndash Communication Coordination

Collaboration

23418 48

bull Ant Colony Optimization (ACO)ndash Inspired by ant colony foragingndash pheromone as heuristic information

(stigmergy)ndash Iteration between ConstructAntSolutions and

UpdatePheromones

bull Particle Swarm Optimization (PSO)ndash Inspired by bird flockingndash Heuristic information results from partnersndash Particle Velocity Update

• Collective Intelligence
• Outline
• The Computational Beauty of Nature
• Examples of Collective Behavior in Nature and Society
• Emergence
• Biological motivation Insect Societies
• Insect Societies
• Self Organization
• Stigmergy
• Flocks Herds and Schools
• Boid rules
• Slide 12
• Swarm Intelligence (Contrsquod)
• Components of SI
• Characteristics of SI
• The Web becomes a Giant Brain Some see the Web evolving into a collective brain for humankind
• What is Multi-agent Systems
• What is MAS(Contrsquod)
• Slide 19
• Communication models
• Working together
• Example mechanism Contract Net Protocol (CNP)
• Slide 23
• CNP (Contrsquod)
• Slide 25
• Attributes of Multi-agent Systems
• Applications of MAS
• Applications of MAS (Contrsquod)
• How to simulate SI for search
• Part Ⅱ Ant Colony Optimization (ACO)
• Natural Ants
• Slide 32
• How can the natural ants find the shortest path
• Slide 34
• Problems of AS
• Part Ⅲ Particle Swarm Optimization (PSO)
• Slide 37
• Modeling bird flocking
• Modeling bird flocking (Contrsquod)
• From Bird to Particle
• Features of PSO
• Particle Swarm Optimization Process
• How do particles fly
• Slide 44
• PSO Related issues
• Advantages of PSO
• Summary
• Slide 48

23418 33

bull They establish indirect communication system based on the deposition of pheromone over the path they followndash An isolated ant moves at random but when it finds a pheromone

trail there is a high probability that this ant will decide to follow the trail

ndash An ant foraging for food deposits pheromone over its route When it finds a food source it returns to the nest reinforcing its trail

ndash So other ants have greater probability to start following this trail and thereby laying more pheromone on it

ndash This process works as a positive feedback loop system because the higher the intensity of the pheromone over a trail the higher the probability of an ant start traveling through it

How can the natural ants find the shortest path

23418 34

bull Since the route B is shorter the ants on this path will complete the travel more times and thereby lay more pheromone over it

bull The pheromone concentration on trail B will increase at a higher rate than on A and soon the ants on route A will choose to follow route B

bull Since most ants will no longer travel on route A and since the pheromone is volatile trail A will start evaporating

bull Only the shortest route will remain

23418 35

Problems of ASbull Ant System tends to converge quickly

ndash This means that its exploitation of the best solution found is too high it should be exploring solution space more

ndash Pheromone evaporationupdate rule (better rule may exist)

bull Led to extensions of the ant systemndash Elitist Strategy for Ant Systems (EAS)ndash Rank based Ant Systems (AKRANK)ndash MAX-MIN Ant system (MMAS)ndash Ant Colony System (ACS)

23418 36

Part Ⅲ Particle Swarm Optimization (PSO)

1048713Firstly Proposed by Kennedy and Eberhart 1995

ldquoOnce again nature has provided us with a technique for processing information that is at once elegant and versatilerdquo

23418 37

bull In PSO a ldquoswarmrdquo is defined as an apparently disorganized collection (population) of moving individuals that tend to cluster together while each individual seems to be moving in a random direction

Bird flocking is one of the best example of PSO in nature

23418 38

Modeling bird flocking

bull The synchrony of flocking behavior is thought to be a function of birdrsquos efforts to maintain an optimal distance between themselves and their neighborsndash Birds and fish adjust their physical movement to avoid

predators seek for food and mates ndash Humans tend to adjust our beliefs and attitudes to

conform with those of our social peers Humans change in abstract multidimensional space collision-free

23418 39

Modeling bird flocking (Contrsquod)

bull Definitions ndash Flock is a group of objects that exhibit the general

class of polarized (aligned) non-colliding aggregate motion

ndash Boid is a simulated bird-like object ie it exhibits this type of behavior It can be a fish bee dinosaur etc

bull Rules for flockingndash Cohesion Each boid fly towards the centroid of its

local flock mates (that is boid in its local neighborhood)ndash Separation Each boid keep a certain distance away

from local flock mates to avoid collisionsndash Alignment Each boid align its velocity vector and keep

velocity magnitude similar with that of the local flock

23418 40

bull Imagine a birdrsquos flock in an area where there is a single food source

bull A bird donrsquot know where the food is but it knows its distance to the food

bull The best strategy is to follow the bird that is closer to the food

bull Particles save and communicate the best solution they have found

From Bird to Particle

23418 41

Features of PSO

bull Population initialized by assigning random positions and velocities potential solutions are then flown through hyperspace

bull Each particle keeps track of its ldquobestrdquo (highest fitness) position in hyperspacendash This is called ldquopBestrdquo for an individual particlendash It is called ldquogBestrdquo for the best in the populationndash It is called ldquolBestrdquo for the best in a defined neighborhood

bull At each time step each particle stochastically accelerates toward its pBest and gBest (or lBest)

23418 42

Particle Swarm Optimization Process

Step1 Initialize population in hyperspace

Step2 Evaluate fitness of individual particles

Step3 Modify velocities based on previous best and global (or neighborhood) best

Step4 Terminate on some condition

Step5 Go to step 2

23418 43

How do particles fly

bull Combination of gBest and the pBest (lBest)ndash need a compromise

bull lBest can bendash Social the particles around are always the same no matter

where they are in spacendash Geographical the particles around are those whose distance is

the shortest

bull Global PSO vs Local PSOndash the global version converges quickly to a solution but it gets

more easily stuck in local minima

23418 44

Illustrating the velocity update schema of global PSO

23418 45

PSO Related issues

bull Controlling velocities (determining the best value for Vmax)ndash Usually set maximum velocity to dynamic range of variable

bull Usually set c1 and c2 to 2

bull Inertia weight influences tradeoff between global and local explorationndash Good approach is to reduce inertia weight during run (ie from

09 to 04 over 1000 generations)

bull Swarm Size and Neighborhood Size

23418 46

Advantages of PSO

bull Adaptation operates on velocitiesndash Most other methods operate on positionsndash Effect PSO has a builtin momentumndash Particles tend to hurdle past optima ndash an advantage

since the best positions are remembered anyway

bull Simple in conceptbull Easy to implementbull Computationally efficientbull Effective on a variety of problems

23418 47

Summarybull Swarm Intelligence (SI)

ndash an artificial intelligence technique based around the study of collective behavior in decentralized self-organized systems

bull Multi-Agent Systems (MAS)ndash A system that consists of a number of agents

which interact with one-anotherndash Communication Coordination

Collaboration

23418 48

bull Ant Colony Optimization (ACO)ndash Inspired by ant colony foragingndash pheromone as heuristic information

(stigmergy)ndash Iteration between ConstructAntSolutions and

UpdatePheromones

bull Particle Swarm Optimization (PSO)ndash Inspired by bird flockingndash Heuristic information results from partnersndash Particle Velocity Update

• Collective Intelligence
• Outline
• The Computational Beauty of Nature
• Examples of Collective Behavior in Nature and Society
• Emergence
• Biological motivation Insect Societies
• Insect Societies
• Self Organization
• Stigmergy
• Flocks Herds and Schools
• Boid rules
• Slide 12
• Swarm Intelligence (Contrsquod)
• Components of SI
• Characteristics of SI
• The Web becomes a Giant Brain Some see the Web evolving into a collective brain for humankind
• What is Multi-agent Systems
• What is MAS(Contrsquod)
• Slide 19
• Communication models
• Working together
• Example mechanism Contract Net Protocol (CNP)
• Slide 23
• CNP (Contrsquod)
• Slide 25
• Attributes of Multi-agent Systems
• Applications of MAS
• Applications of MAS (Contrsquod)
• How to simulate SI for search
• Part Ⅱ Ant Colony Optimization (ACO)
• Natural Ants
• Slide 32
• How can the natural ants find the shortest path
• Slide 34
• Problems of AS
• Part Ⅲ Particle Swarm Optimization (PSO)
• Slide 37
• Modeling bird flocking
• Modeling bird flocking (Contrsquod)
• From Bird to Particle
• Features of PSO
• Particle Swarm Optimization Process
• How do particles fly
• Slide 44
• PSO Related issues
• Advantages of PSO
• Summary
• Slide 48

23418 34

bull Since the route B is shorter the ants on this path will complete the travel more times and thereby lay more pheromone over it

bull The pheromone concentration on trail B will increase at a higher rate than on A and soon the ants on route A will choose to follow route B

bull Since most ants will no longer travel on route A and since the pheromone is volatile trail A will start evaporating

bull Only the shortest route will remain

23418 35

Problems of ASbull Ant System tends to converge quickly

ndash This means that its exploitation of the best solution found is too high it should be exploring solution space more

ndash Pheromone evaporationupdate rule (better rule may exist)

bull Led to extensions of the ant systemndash Elitist Strategy for Ant Systems (EAS)ndash Rank based Ant Systems (AKRANK)ndash MAX-MIN Ant system (MMAS)ndash Ant Colony System (ACS)

23418 36

Part Ⅲ Particle Swarm Optimization (PSO)

1048713Firstly Proposed by Kennedy and Eberhart 1995

ldquoOnce again nature has provided us with a technique for processing information that is at once elegant and versatilerdquo

23418 37

bull In PSO a ldquoswarmrdquo is defined as an apparently disorganized collection (population) of moving individuals that tend to cluster together while each individual seems to be moving in a random direction

Bird flocking is one of the best example of PSO in nature

23418 38

Modeling bird flocking

bull The synchrony of flocking behavior is thought to be a function of birdrsquos efforts to maintain an optimal distance between themselves and their neighborsndash Birds and fish adjust their physical movement to avoid

predators seek for food and mates ndash Humans tend to adjust our beliefs and attitudes to

conform with those of our social peers Humans change in abstract multidimensional space collision-free

23418 39

Modeling bird flocking (Contrsquod)

bull Definitions ndash Flock is a group of objects that exhibit the general

class of polarized (aligned) non-colliding aggregate motion

ndash Boid is a simulated bird-like object ie it exhibits this type of behavior It can be a fish bee dinosaur etc

bull Rules for flockingndash Cohesion Each boid fly towards the centroid of its

local flock mates (that is boid in its local neighborhood)ndash Separation Each boid keep a certain distance away

from local flock mates to avoid collisionsndash Alignment Each boid align its velocity vector and keep

velocity magnitude similar with that of the local flock

23418 40

bull Imagine a birdrsquos flock in an area where there is a single food source

bull A bird donrsquot know where the food is but it knows its distance to the food

bull The best strategy is to follow the bird that is closer to the food

bull Particles save and communicate the best solution they have found

From Bird to Particle

23418 41

Features of PSO

bull Population initialized by assigning random positions and velocities potential solutions are then flown through hyperspace

bull Each particle keeps track of its ldquobestrdquo (highest fitness) position in hyperspacendash This is called ldquopBestrdquo for an individual particlendash It is called ldquogBestrdquo for the best in the populationndash It is called ldquolBestrdquo for the best in a defined neighborhood

bull At each time step each particle stochastically accelerates toward its pBest and gBest (or lBest)

23418 42

Particle Swarm Optimization Process

Step1 Initialize population in hyperspace

Step2 Evaluate fitness of individual particles

Step3 Modify velocities based on previous best and global (or neighborhood) best

Step4 Terminate on some condition

Step5 Go to step 2

23418 43

How do particles fly

bull Combination of gBest and the pBest (lBest)ndash need a compromise

bull lBest can bendash Social the particles around are always the same no matter

where they are in spacendash Geographical the particles around are those whose distance is

the shortest

bull Global PSO vs Local PSOndash the global version converges quickly to a solution but it gets

more easily stuck in local minima

23418 44

Illustrating the velocity update schema of global PSO

23418 45

PSO Related issues

bull Controlling velocities (determining the best value for Vmax)ndash Usually set maximum velocity to dynamic range of variable

bull Usually set c1 and c2 to 2

bull Inertia weight influences tradeoff between global and local explorationndash Good approach is to reduce inertia weight during run (ie from

09 to 04 over 1000 generations)

bull Swarm Size and Neighborhood Size

23418 46

Advantages of PSO

bull Adaptation operates on velocitiesndash Most other methods operate on positionsndash Effect PSO has a builtin momentumndash Particles tend to hurdle past optima ndash an advantage

since the best positions are remembered anyway

bull Simple in conceptbull Easy to implementbull Computationally efficientbull Effective on a variety of problems

23418 47

Summarybull Swarm Intelligence (SI)

ndash an artificial intelligence technique based around the study of collective behavior in decentralized self-organized systems

bull Multi-Agent Systems (MAS)ndash A system that consists of a number of agents

which interact with one-anotherndash Communication Coordination

Collaboration

23418 48

bull Ant Colony Optimization (ACO)ndash Inspired by ant colony foragingndash pheromone as heuristic information

(stigmergy)ndash Iteration between ConstructAntSolutions and

UpdatePheromones

bull Particle Swarm Optimization (PSO)ndash Inspired by bird flockingndash Heuristic information results from partnersndash Particle Velocity Update

• Collective Intelligence
• Outline
• The Computational Beauty of Nature
• Examples of Collective Behavior in Nature and Society
• Emergence
• Biological motivation Insect Societies
• Insect Societies
• Self Organization
• Stigmergy
• Flocks Herds and Schools
• Boid rules
• Slide 12
• Swarm Intelligence (Contrsquod)
• Components of SI
• Characteristics of SI
• The Web becomes a Giant Brain Some see the Web evolving into a collective brain for humankind
• What is Multi-agent Systems
• What is MAS(Contrsquod)
• Slide 19
• Communication models
• Working together
• Example mechanism Contract Net Protocol (CNP)
• Slide 23
• CNP (Contrsquod)
• Slide 25
• Attributes of Multi-agent Systems
• Applications of MAS
• Applications of MAS (Contrsquod)
• How to simulate SI for search
• Part Ⅱ Ant Colony Optimization (ACO)
• Natural Ants
• Slide 32
• How can the natural ants find the shortest path
• Slide 34
• Problems of AS
• Part Ⅲ Particle Swarm Optimization (PSO)
• Slide 37
• Modeling bird flocking
• Modeling bird flocking (Contrsquod)
• From Bird to Particle
• Features of PSO
• Particle Swarm Optimization Process
• How do particles fly
• Slide 44
• PSO Related issues
• Advantages of PSO
• Summary
• Slide 48

23418 35

Problems of ASbull Ant System tends to converge quickly

ndash This means that its exploitation of the best solution found is too high it should be exploring solution space more

ndash Pheromone evaporationupdate rule (better rule may exist)

bull Led to extensions of the ant systemndash Elitist Strategy for Ant Systems (EAS)ndash Rank based Ant Systems (AKRANK)ndash MAX-MIN Ant system (MMAS)ndash Ant Colony System (ACS)

23418 36

Part Ⅲ Particle Swarm Optimization (PSO)

1048713Firstly Proposed by Kennedy and Eberhart 1995

ldquoOnce again nature has provided us with a technique for processing information that is at once elegant and versatilerdquo

23418 37

bull In PSO a ldquoswarmrdquo is defined as an apparently disorganized collection (population) of moving individuals that tend to cluster together while each individual seems to be moving in a random direction

Bird flocking is one of the best example of PSO in nature

23418 38

Modeling bird flocking

bull The synchrony of flocking behavior is thought to be a function of birdrsquos efforts to maintain an optimal distance between themselves and their neighborsndash Birds and fish adjust their physical movement to avoid

predators seek for food and mates ndash Humans tend to adjust our beliefs and attitudes to

conform with those of our social peers Humans change in abstract multidimensional space collision-free

23418 39

Modeling bird flocking (Contrsquod)

bull Definitions ndash Flock is a group of objects that exhibit the general

class of polarized (aligned) non-colliding aggregate motion

ndash Boid is a simulated bird-like object ie it exhibits this type of behavior It can be a fish bee dinosaur etc

bull Rules for flockingndash Cohesion Each boid fly towards the centroid of its

local flock mates (that is boid in its local neighborhood)ndash Separation Each boid keep a certain distance away

from local flock mates to avoid collisionsndash Alignment Each boid align its velocity vector and keep

velocity magnitude similar with that of the local flock

23418 40

bull Imagine a birdrsquos flock in an area where there is a single food source

bull A bird donrsquot know where the food is but it knows its distance to the food

bull The best strategy is to follow the bird that is closer to the food

bull Particles save and communicate the best solution they have found

From Bird to Particle

23418 41

Features of PSO

bull Population initialized by assigning random positions and velocities potential solutions are then flown through hyperspace

bull Each particle keeps track of its ldquobestrdquo (highest fitness) position in hyperspacendash This is called ldquopBestrdquo for an individual particlendash It is called ldquogBestrdquo for the best in the populationndash It is called ldquolBestrdquo for the best in a defined neighborhood

bull At each time step each particle stochastically accelerates toward its pBest and gBest (or lBest)

23418 42

Particle Swarm Optimization Process

Step1 Initialize population in hyperspace

Step2 Evaluate fitness of individual particles

Step3 Modify velocities based on previous best and global (or neighborhood) best

Step4 Terminate on some condition

Step5 Go to step 2

23418 43

How do particles fly

bull Combination of gBest and the pBest (lBest)ndash need a compromise

bull lBest can bendash Social the particles around are always the same no matter

where they are in spacendash Geographical the particles around are those whose distance is

the shortest

bull Global PSO vs Local PSOndash the global version converges quickly to a solution but it gets

more easily stuck in local minima

23418 44

Illustrating the velocity update schema of global PSO

23418 45

PSO Related issues

bull Controlling velocities (determining the best value for Vmax)ndash Usually set maximum velocity to dynamic range of variable

bull Usually set c1 and c2 to 2

bull Inertia weight influences tradeoff between global and local explorationndash Good approach is to reduce inertia weight during run (ie from

09 to 04 over 1000 generations)

bull Swarm Size and Neighborhood Size

23418 46

Advantages of PSO

bull Adaptation operates on velocitiesndash Most other methods operate on positionsndash Effect PSO has a builtin momentumndash Particles tend to hurdle past optima ndash an advantage

since the best positions are remembered anyway

bull Simple in conceptbull Easy to implementbull Computationally efficientbull Effective on a variety of problems

23418 47

Summarybull Swarm Intelligence (SI)

ndash an artificial intelligence technique based around the study of collective behavior in decentralized self-organized systems

bull Multi-Agent Systems (MAS)ndash A system that consists of a number of agents

which interact with one-anotherndash Communication Coordination

Collaboration

23418 48

bull Ant Colony Optimization (ACO)ndash Inspired by ant colony foragingndash pheromone as heuristic information

(stigmergy)ndash Iteration between ConstructAntSolutions and

UpdatePheromones

bull Particle Swarm Optimization (PSO)ndash Inspired by bird flockingndash Heuristic information results from partnersndash Particle Velocity Update

• Collective Intelligence
• Outline
• The Computational Beauty of Nature
• Examples of Collective Behavior in Nature and Society
• Emergence
• Biological motivation Insect Societies
• Insect Societies
• Self Organization
• Stigmergy
• Flocks Herds and Schools
• Boid rules
• Slide 12
• Swarm Intelligence (Contrsquod)
• Components of SI
• Characteristics of SI
• The Web becomes a Giant Brain Some see the Web evolving into a collective brain for humankind
• What is Multi-agent Systems
• What is MAS(Contrsquod)
• Slide 19
• Communication models
• Working together
• Example mechanism Contract Net Protocol (CNP)
• Slide 23
• CNP (Contrsquod)
• Slide 25
• Attributes of Multi-agent Systems
• Applications of MAS
• Applications of MAS (Contrsquod)
• How to simulate SI for search
• Part Ⅱ Ant Colony Optimization (ACO)
• Natural Ants
• Slide 32
• How can the natural ants find the shortest path
• Slide 34
• Problems of AS
• Part Ⅲ Particle Swarm Optimization (PSO)
• Slide 37
• Modeling bird flocking
• Modeling bird flocking (Contrsquod)
• From Bird to Particle
• Features of PSO
• Particle Swarm Optimization Process
• How do particles fly
• Slide 44
• PSO Related issues
• Advantages of PSO
• Summary
• Slide 48

23418 36

Part Ⅲ Particle Swarm Optimization (PSO)

1048713Firstly Proposed by Kennedy and Eberhart 1995

ldquoOnce again nature has provided us with a technique for processing information that is at once elegant and versatilerdquo

23418 37

bull In PSO a ldquoswarmrdquo is defined as an apparently disorganized collection (population) of moving individuals that tend to cluster together while each individual seems to be moving in a random direction

Bird flocking is one of the best example of PSO in nature

23418 38

Modeling bird flocking

bull The synchrony of flocking behavior is thought to be a function of birdrsquos efforts to maintain an optimal distance between themselves and their neighborsndash Birds and fish adjust their physical movement to avoid

predators seek for food and mates ndash Humans tend to adjust our beliefs and attitudes to

conform with those of our social peers Humans change in abstract multidimensional space collision-free

23418 39

Modeling bird flocking (Contrsquod)

bull Definitions ndash Flock is a group of objects that exhibit the general

class of polarized (aligned) non-colliding aggregate motion

ndash Boid is a simulated bird-like object ie it exhibits this type of behavior It can be a fish bee dinosaur etc

bull Rules for flockingndash Cohesion Each boid fly towards the centroid of its

local flock mates (that is boid in its local neighborhood)ndash Separation Each boid keep a certain distance away

from local flock mates to avoid collisionsndash Alignment Each boid align its velocity vector and keep

velocity magnitude similar with that of the local flock

23418 40

bull Imagine a birdrsquos flock in an area where there is a single food source

bull A bird donrsquot know where the food is but it knows its distance to the food

bull The best strategy is to follow the bird that is closer to the food

bull Particles save and communicate the best solution they have found

From Bird to Particle

23418 41

Features of PSO

bull Population initialized by assigning random positions and velocities potential solutions are then flown through hyperspace

bull Each particle keeps track of its ldquobestrdquo (highest fitness) position in hyperspacendash This is called ldquopBestrdquo for an individual particlendash It is called ldquogBestrdquo for the best in the populationndash It is called ldquolBestrdquo for the best in a defined neighborhood

bull At each time step each particle stochastically accelerates toward its pBest and gBest (or lBest)

23418 42

Particle Swarm Optimization Process

Step1 Initialize population in hyperspace

Step2 Evaluate fitness of individual particles

Step3 Modify velocities based on previous best and global (or neighborhood) best

Step4 Terminate on some condition

Step5 Go to step 2

23418 43

How do particles fly

bull Combination of gBest and the pBest (lBest)ndash need a compromise

bull lBest can bendash Social the particles around are always the same no matter

where they are in spacendash Geographical the particles around are those whose distance is

the shortest

bull Global PSO vs Local PSOndash the global version converges quickly to a solution but it gets

more easily stuck in local minima

23418 44

Illustrating the velocity update schema of global PSO

23418 45

PSO Related issues

bull Controlling velocities (determining the best value for Vmax)ndash Usually set maximum velocity to dynamic range of variable

bull Usually set c1 and c2 to 2

bull Inertia weight influences tradeoff between global and local explorationndash Good approach is to reduce inertia weight during run (ie from

09 to 04 over 1000 generations)

bull Swarm Size and Neighborhood Size

23418 46

Advantages of PSO

bull Adaptation operates on velocitiesndash Most other methods operate on positionsndash Effect PSO has a builtin momentumndash Particles tend to hurdle past optima ndash an advantage

since the best positions are remembered anyway

bull Simple in conceptbull Easy to implementbull Computationally efficientbull Effective on a variety of problems

23418 47

Summarybull Swarm Intelligence (SI)

ndash an artificial intelligence technique based around the study of collective behavior in decentralized self-organized systems

bull Multi-Agent Systems (MAS)ndash A system that consists of a number of agents

which interact with one-anotherndash Communication Coordination

Collaboration

23418 48

bull Ant Colony Optimization (ACO)ndash Inspired by ant colony foragingndash pheromone as heuristic information

(stigmergy)ndash Iteration between ConstructAntSolutions and

UpdatePheromones

bull Particle Swarm Optimization (PSO)ndash Inspired by bird flockingndash Heuristic information results from partnersndash Particle Velocity Update

• Collective Intelligence
• Outline
• The Computational Beauty of Nature
• Examples of Collective Behavior in Nature and Society
• Emergence
• Biological motivation Insect Societies
• Insect Societies
• Self Organization
• Stigmergy
• Flocks Herds and Schools
• Boid rules
• Slide 12
• Swarm Intelligence (Contrsquod)
• Components of SI
• Characteristics of SI
• The Web becomes a Giant Brain Some see the Web evolving into a collective brain for humankind
• What is Multi-agent Systems
• What is MAS(Contrsquod)
• Slide 19
• Communication models
• Working together
• Example mechanism Contract Net Protocol (CNP)
• Slide 23
• CNP (Contrsquod)
• Slide 25
• Attributes of Multi-agent Systems
• Applications of MAS
• Applications of MAS (Contrsquod)
• How to simulate SI for search
• Part Ⅱ Ant Colony Optimization (ACO)
• Natural Ants
• Slide 32
• How can the natural ants find the shortest path
• Slide 34
• Problems of AS
• Part Ⅲ Particle Swarm Optimization (PSO)
• Slide 37
• Modeling bird flocking
• Modeling bird flocking (Contrsquod)
• From Bird to Particle
• Features of PSO
• Particle Swarm Optimization Process
• How do particles fly
• Slide 44
• PSO Related issues
• Advantages of PSO
• Summary
• Slide 48

23418 37

bull In PSO a ldquoswarmrdquo is defined as an apparently disorganized collection (population) of moving individuals that tend to cluster together while each individual seems to be moving in a random direction

Bird flocking is one of the best example of PSO in nature

23418 38

Modeling bird flocking

bull The synchrony of flocking behavior is thought to be a function of birdrsquos efforts to maintain an optimal distance between themselves and their neighborsndash Birds and fish adjust their physical movement to avoid

predators seek for food and mates ndash Humans tend to adjust our beliefs and attitudes to

conform with those of our social peers Humans change in abstract multidimensional space collision-free

23418 39

Modeling bird flocking (Contrsquod)

bull Definitions ndash Flock is a group of objects that exhibit the general

class of polarized (aligned) non-colliding aggregate motion

ndash Boid is a simulated bird-like object ie it exhibits this type of behavior It can be a fish bee dinosaur etc

bull Rules for flockingndash Cohesion Each boid fly towards the centroid of its

local flock mates (that is boid in its local neighborhood)ndash Separation Each boid keep a certain distance away

from local flock mates to avoid collisionsndash Alignment Each boid align its velocity vector and keep

velocity magnitude similar with that of the local flock

23418 40

bull Imagine a birdrsquos flock in an area where there is a single food source

bull A bird donrsquot know where the food is but it knows its distance to the food

bull The best strategy is to follow the bird that is closer to the food

bull Particles save and communicate the best solution they have found

From Bird to Particle

23418 41

Features of PSO

bull Population initialized by assigning random positions and velocities potential solutions are then flown through hyperspace

bull Each particle keeps track of its ldquobestrdquo (highest fitness) position in hyperspacendash This is called ldquopBestrdquo for an individual particlendash It is called ldquogBestrdquo for the best in the populationndash It is called ldquolBestrdquo for the best in a defined neighborhood

bull At each time step each particle stochastically accelerates toward its pBest and gBest (or lBest)

23418 42

Particle Swarm Optimization Process

Step1 Initialize population in hyperspace

Step2 Evaluate fitness of individual particles

Step3 Modify velocities based on previous best and global (or neighborhood) best

Step4 Terminate on some condition

Step5 Go to step 2

23418 43

How do particles fly

bull Combination of gBest and the pBest (lBest)ndash need a compromise

bull lBest can bendash Social the particles around are always the same no matter

where they are in spacendash Geographical the particles around are those whose distance is

the shortest

bull Global PSO vs Local PSOndash the global version converges quickly to a solution but it gets

more easily stuck in local minima

23418 44

Illustrating the velocity update schema of global PSO

23418 45

PSO Related issues

bull Controlling velocities (determining the best value for Vmax)ndash Usually set maximum velocity to dynamic range of variable

bull Usually set c1 and c2 to 2

bull Inertia weight influences tradeoff between global and local explorationndash Good approach is to reduce inertia weight during run (ie from

09 to 04 over 1000 generations)

bull Swarm Size and Neighborhood Size

23418 46

Advantages of PSO

bull Adaptation operates on velocitiesndash Most other methods operate on positionsndash Effect PSO has a builtin momentumndash Particles tend to hurdle past optima ndash an advantage

since the best positions are remembered anyway

bull Simple in conceptbull Easy to implementbull Computationally efficientbull Effective on a variety of problems

23418 47

Summarybull Swarm Intelligence (SI)

ndash an artificial intelligence technique based around the study of collective behavior in decentralized self-organized systems

bull Multi-Agent Systems (MAS)ndash A system that consists of a number of agents

which interact with one-anotherndash Communication Coordination

Collaboration

23418 48

bull Ant Colony Optimization (ACO)ndash Inspired by ant colony foragingndash pheromone as heuristic information

(stigmergy)ndash Iteration between ConstructAntSolutions and

UpdatePheromones

bull Particle Swarm Optimization (PSO)ndash Inspired by bird flockingndash Heuristic information results from partnersndash Particle Velocity Update

• Collective Intelligence
• Outline
• The Computational Beauty of Nature
• Examples of Collective Behavior in Nature and Society
• Emergence
• Biological motivation Insect Societies
• Insect Societies
• Self Organization
• Stigmergy
• Flocks Herds and Schools
• Boid rules
• Slide 12
• Swarm Intelligence (Contrsquod)
• Components of SI
• Characteristics of SI
• The Web becomes a Giant Brain Some see the Web evolving into a collective brain for humankind
• What is Multi-agent Systems
• What is MAS(Contrsquod)
• Slide 19
• Communication models
• Working together
• Example mechanism Contract Net Protocol (CNP)
• Slide 23
• CNP (Contrsquod)
• Slide 25
• Attributes of Multi-agent Systems
• Applications of MAS
• Applications of MAS (Contrsquod)
• How to simulate SI for search
• Part Ⅱ Ant Colony Optimization (ACO)
• Natural Ants
• Slide 32
• How can the natural ants find the shortest path
• Slide 34
• Problems of AS
• Part Ⅲ Particle Swarm Optimization (PSO)
• Slide 37
• Modeling bird flocking
• Modeling bird flocking (Contrsquod)
• From Bird to Particle
• Features of PSO
• Particle Swarm Optimization Process
• How do particles fly
• Slide 44
• PSO Related issues
• Advantages of PSO
• Summary
• Slide 48

23418 38

Modeling bird flocking

bull The synchrony of flocking behavior is thought to be a function of birdrsquos efforts to maintain an optimal distance between themselves and their neighborsndash Birds and fish adjust their physical movement to avoid

predators seek for food and mates ndash Humans tend to adjust our beliefs and attitudes to

conform with those of our social peers Humans change in abstract multidimensional space collision-free

23418 39

Modeling bird flocking (Contrsquod)

bull Definitions ndash Flock is a group of objects that exhibit the general

class of polarized (aligned) non-colliding aggregate motion

ndash Boid is a simulated bird-like object ie it exhibits this type of behavior It can be a fish bee dinosaur etc

bull Rules for flockingndash Cohesion Each boid fly towards the centroid of its

local flock mates (that is boid in its local neighborhood)ndash Separation Each boid keep a certain distance away

from local flock mates to avoid collisionsndash Alignment Each boid align its velocity vector and keep

velocity magnitude similar with that of the local flock

23418 40

bull Imagine a birdrsquos flock in an area where there is a single food source

bull A bird donrsquot know where the food is but it knows its distance to the food

bull The best strategy is to follow the bird that is closer to the food

bull Particles save and communicate the best solution they have found

From Bird to Particle

23418 41

Features of PSO

bull Population initialized by assigning random positions and velocities potential solutions are then flown through hyperspace

bull Each particle keeps track of its ldquobestrdquo (highest fitness) position in hyperspacendash This is called ldquopBestrdquo for an individual particlendash It is called ldquogBestrdquo for the best in the populationndash It is called ldquolBestrdquo for the best in a defined neighborhood

bull At each time step each particle stochastically accelerates toward its pBest and gBest (or lBest)

23418 42

Particle Swarm Optimization Process

Step1 Initialize population in hyperspace

Step2 Evaluate fitness of individual particles

Step3 Modify velocities based on previous best and global (or neighborhood) best

Step4 Terminate on some condition

Step5 Go to step 2

23418 43

How do particles fly

bull Combination of gBest and the pBest (lBest)ndash need a compromise

bull lBest can bendash Social the particles around are always the same no matter

where they are in spacendash Geographical the particles around are those whose distance is

the shortest

bull Global PSO vs Local PSOndash the global version converges quickly to a solution but it gets

more easily stuck in local minima

23418 44

Illustrating the velocity update schema of global PSO

23418 45

PSO Related issues

bull Controlling velocities (determining the best value for Vmax)ndash Usually set maximum velocity to dynamic range of variable

bull Usually set c1 and c2 to 2

bull Inertia weight influences tradeoff between global and local explorationndash Good approach is to reduce inertia weight during run (ie from

09 to 04 over 1000 generations)

bull Swarm Size and Neighborhood Size

23418 46

Advantages of PSO

bull Adaptation operates on velocitiesndash Most other methods operate on positionsndash Effect PSO has a builtin momentumndash Particles tend to hurdle past optima ndash an advantage

since the best positions are remembered anyway

bull Simple in conceptbull Easy to implementbull Computationally efficientbull Effective on a variety of problems

23418 47

Summarybull Swarm Intelligence (SI)

ndash an artificial intelligence technique based around the study of collective behavior in decentralized self-organized systems

bull Multi-Agent Systems (MAS)ndash A system that consists of a number of agents

which interact with one-anotherndash Communication Coordination

Collaboration

23418 48

bull Ant Colony Optimization (ACO)ndash Inspired by ant colony foragingndash pheromone as heuristic information

(stigmergy)ndash Iteration between ConstructAntSolutions and

UpdatePheromones

bull Particle Swarm Optimization (PSO)ndash Inspired by bird flockingndash Heuristic information results from partnersndash Particle Velocity Update

• Collective Intelligence
• Outline
• The Computational Beauty of Nature
• Examples of Collective Behavior in Nature and Society
• Emergence
• Biological motivation Insect Societies
• Insect Societies
• Self Organization
• Stigmergy
• Flocks Herds and Schools
• Boid rules
• Slide 12
• Swarm Intelligence (Contrsquod)
• Components of SI
• Characteristics of SI
• The Web becomes a Giant Brain Some see the Web evolving into a collective brain for humankind
• What is Multi-agent Systems
• What is MAS(Contrsquod)
• Slide 19
• Communication models
• Working together
• Example mechanism Contract Net Protocol (CNP)
• Slide 23
• CNP (Contrsquod)
• Slide 25
• Attributes of Multi-agent Systems
• Applications of MAS
• Applications of MAS (Contrsquod)
• How to simulate SI for search
• Part Ⅱ Ant Colony Optimization (ACO)
• Natural Ants
• Slide 32
• How can the natural ants find the shortest path
• Slide 34
• Problems of AS
• Part Ⅲ Particle Swarm Optimization (PSO)
• Slide 37
• Modeling bird flocking
• Modeling bird flocking (Contrsquod)
• From Bird to Particle
• Features of PSO
• Particle Swarm Optimization Process
• How do particles fly
• Slide 44
• PSO Related issues
• Advantages of PSO
• Summary
• Slide 48

23418 39

Modeling bird flocking (Contrsquod)

bull Definitions ndash Flock is a group of objects that exhibit the general

class of polarized (aligned) non-colliding aggregate motion

ndash Boid is a simulated bird-like object ie it exhibits this type of behavior It can be a fish bee dinosaur etc

bull Rules for flockingndash Cohesion Each boid fly towards the centroid of its

local flock mates (that is boid in its local neighborhood)ndash Separation Each boid keep a certain distance away

from local flock mates to avoid collisionsndash Alignment Each boid align its velocity vector and keep

velocity magnitude similar with that of the local flock

23418 40

bull Imagine a birdrsquos flock in an area where there is a single food source

bull A bird donrsquot know where the food is but it knows its distance to the food

bull The best strategy is to follow the bird that is closer to the food

bull Particles save and communicate the best solution they have found

From Bird to Particle

23418 41

Features of PSO

bull Population initialized by assigning random positions and velocities potential solutions are then flown through hyperspace

bull Each particle keeps track of its ldquobestrdquo (highest fitness) position in hyperspacendash This is called ldquopBestrdquo for an individual particlendash It is called ldquogBestrdquo for the best in the populationndash It is called ldquolBestrdquo for the best in a defined neighborhood

bull At each time step each particle stochastically accelerates toward its pBest and gBest (or lBest)

23418 42

Particle Swarm Optimization Process

Step1 Initialize population in hyperspace

Step2 Evaluate fitness of individual particles

Step3 Modify velocities based on previous best and global (or neighborhood) best

Step4 Terminate on some condition

Step5 Go to step 2

23418 43

How do particles fly

bull Combination of gBest and the pBest (lBest)ndash need a compromise

bull lBest can bendash Social the particles around are always the same no matter

where they are in spacendash Geographical the particles around are those whose distance is

the shortest

bull Global PSO vs Local PSOndash the global version converges quickly to a solution but it gets

more easily stuck in local minima

23418 44

Illustrating the velocity update schema of global PSO

23418 45

PSO Related issues

bull Controlling velocities (determining the best value for Vmax)ndash Usually set maximum velocity to dynamic range of variable

bull Usually set c1 and c2 to 2

bull Inertia weight influences tradeoff between global and local explorationndash Good approach is to reduce inertia weight during run (ie from

09 to 04 over 1000 generations)

bull Swarm Size and Neighborhood Size

23418 46

Advantages of PSO

bull Adaptation operates on velocitiesndash Most other methods operate on positionsndash Effect PSO has a builtin momentumndash Particles tend to hurdle past optima ndash an advantage

since the best positions are remembered anyway

bull Simple in conceptbull Easy to implementbull Computationally efficientbull Effective on a variety of problems

23418 47

Summarybull Swarm Intelligence (SI)

ndash an artificial intelligence technique based around the study of collective behavior in decentralized self-organized systems

bull Multi-Agent Systems (MAS)ndash A system that consists of a number of agents

which interact with one-anotherndash Communication Coordination

Collaboration

23418 48

bull Ant Colony Optimization (ACO)ndash Inspired by ant colony foragingndash pheromone as heuristic information

(stigmergy)ndash Iteration between ConstructAntSolutions and

UpdatePheromones

bull Particle Swarm Optimization (PSO)ndash Inspired by bird flockingndash Heuristic information results from partnersndash Particle Velocity Update

• Collective Intelligence
• Outline
• The Computational Beauty of Nature
• Examples of Collective Behavior in Nature and Society
• Emergence
• Biological motivation Insect Societies
• Insect Societies
• Self Organization
• Stigmergy
• Flocks Herds and Schools
• Boid rules
• Slide 12
• Swarm Intelligence (Contrsquod)
• Components of SI
• Characteristics of SI
• The Web becomes a Giant Brain Some see the Web evolving into a collective brain for humankind
• What is Multi-agent Systems
• What is MAS(Contrsquod)
• Slide 19
• Communication models
• Working together
• Example mechanism Contract Net Protocol (CNP)
• Slide 23
• CNP (Contrsquod)
• Slide 25
• Attributes of Multi-agent Systems
• Applications of MAS
• Applications of MAS (Contrsquod)
• How to simulate SI for search
• Part Ⅱ Ant Colony Optimization (ACO)
• Natural Ants
• Slide 32
• How can the natural ants find the shortest path
• Slide 34
• Problems of AS
• Part Ⅲ Particle Swarm Optimization (PSO)
• Slide 37
• Modeling bird flocking
• Modeling bird flocking (Contrsquod)
• From Bird to Particle
• Features of PSO
• Particle Swarm Optimization Process
• How do particles fly
• Slide 44
• PSO Related issues
• Advantages of PSO
• Summary
• Slide 48

23418 40

bull Imagine a birdrsquos flock in an area where there is a single food source

bull A bird donrsquot know where the food is but it knows its distance to the food

bull The best strategy is to follow the bird that is closer to the food

bull Particles save and communicate the best solution they have found

From Bird to Particle

23418 41

Features of PSO

bull Population initialized by assigning random positions and velocities potential solutions are then flown through hyperspace

bull Each particle keeps track of its ldquobestrdquo (highest fitness) position in hyperspacendash This is called ldquopBestrdquo for an individual particlendash It is called ldquogBestrdquo for the best in the populationndash It is called ldquolBestrdquo for the best in a defined neighborhood

bull At each time step each particle stochastically accelerates toward its pBest and gBest (or lBest)

23418 42

Particle Swarm Optimization Process

Step1 Initialize population in hyperspace

Step2 Evaluate fitness of individual particles

Step3 Modify velocities based on previous best and global (or neighborhood) best

Step4 Terminate on some condition

Step5 Go to step 2

23418 43

How do particles fly

bull Combination of gBest and the pBest (lBest)ndash need a compromise

bull lBest can bendash Social the particles around are always the same no matter

where they are in spacendash Geographical the particles around are those whose distance is

the shortest

bull Global PSO vs Local PSOndash the global version converges quickly to a solution but it gets

more easily stuck in local minima

23418 44

Illustrating the velocity update schema of global PSO

23418 45

PSO Related issues

bull Controlling velocities (determining the best value for Vmax)ndash Usually set maximum velocity to dynamic range of variable

bull Usually set c1 and c2 to 2

bull Inertia weight influences tradeoff between global and local explorationndash Good approach is to reduce inertia weight during run (ie from

09 to 04 over 1000 generations)

bull Swarm Size and Neighborhood Size

23418 46

Advantages of PSO

bull Adaptation operates on velocitiesndash Most other methods operate on positionsndash Effect PSO has a builtin momentumndash Particles tend to hurdle past optima ndash an advantage

since the best positions are remembered anyway

bull Simple in conceptbull Easy to implementbull Computationally efficientbull Effective on a variety of problems

23418 47

Summarybull Swarm Intelligence (SI)

ndash an artificial intelligence technique based around the study of collective behavior in decentralized self-organized systems

bull Multi-Agent Systems (MAS)ndash A system that consists of a number of agents

which interact with one-anotherndash Communication Coordination

Collaboration

23418 48

bull Ant Colony Optimization (ACO)ndash Inspired by ant colony foragingndash pheromone as heuristic information

(stigmergy)ndash Iteration between ConstructAntSolutions and

UpdatePheromones

bull Particle Swarm Optimization (PSO)ndash Inspired by bird flockingndash Heuristic information results from partnersndash Particle Velocity Update

• Collective Intelligence
• Outline
• The Computational Beauty of Nature
• Examples of Collective Behavior in Nature and Society
• Emergence
• Biological motivation Insect Societies
• Insect Societies
• Self Organization
• Stigmergy
• Flocks Herds and Schools
• Boid rules
• Slide 12
• Swarm Intelligence (Contrsquod)
• Components of SI
• Characteristics of SI
• The Web becomes a Giant Brain Some see the Web evolving into a collective brain for humankind
• What is Multi-agent Systems
• What is MAS(Contrsquod)
• Slide 19
• Communication models
• Working together
• Example mechanism Contract Net Protocol (CNP)
• Slide 23
• CNP (Contrsquod)
• Slide 25
• Attributes of Multi-agent Systems
• Applications of MAS
• Applications of MAS (Contrsquod)
• How to simulate SI for search
• Part Ⅱ Ant Colony Optimization (ACO)
• Natural Ants
• Slide 32
• How can the natural ants find the shortest path
• Slide 34
• Problems of AS
• Part Ⅲ Particle Swarm Optimization (PSO)
• Slide 37
• Modeling bird flocking
• Modeling bird flocking (Contrsquod)
• From Bird to Particle
• Features of PSO
• Particle Swarm Optimization Process
• How do particles fly
• Slide 44
• PSO Related issues
• Advantages of PSO
• Summary
• Slide 48

23418 41

Features of PSO

bull Population initialized by assigning random positions and velocities potential solutions are then flown through hyperspace

bull Each particle keeps track of its ldquobestrdquo (highest fitness) position in hyperspacendash This is called ldquopBestrdquo for an individual particlendash It is called ldquogBestrdquo for the best in the populationndash It is called ldquolBestrdquo for the best in a defined neighborhood

bull At each time step each particle stochastically accelerates toward its pBest and gBest (or lBest)

23418 42

Particle Swarm Optimization Process

Step1 Initialize population in hyperspace

Step2 Evaluate fitness of individual particles

Step3 Modify velocities based on previous best and global (or neighborhood) best

Step4 Terminate on some condition

Step5 Go to step 2

23418 43

How do particles fly

bull Combination of gBest and the pBest (lBest)ndash need a compromise

bull lBest can bendash Social the particles around are always the same no matter

where they are in spacendash Geographical the particles around are those whose distance is

the shortest

bull Global PSO vs Local PSOndash the global version converges quickly to a solution but it gets

more easily stuck in local minima

23418 44

Illustrating the velocity update schema of global PSO

23418 45

PSO Related issues

bull Controlling velocities (determining the best value for Vmax)ndash Usually set maximum velocity to dynamic range of variable

bull Usually set c1 and c2 to 2

bull Inertia weight influences tradeoff between global and local explorationndash Good approach is to reduce inertia weight during run (ie from

09 to 04 over 1000 generations)

bull Swarm Size and Neighborhood Size

23418 46

Advantages of PSO

bull Adaptation operates on velocitiesndash Most other methods operate on positionsndash Effect PSO has a builtin momentumndash Particles tend to hurdle past optima ndash an advantage

since the best positions are remembered anyway

bull Simple in conceptbull Easy to implementbull Computationally efficientbull Effective on a variety of problems

23418 47

Summarybull Swarm Intelligence (SI)

ndash an artificial intelligence technique based around the study of collective behavior in decentralized self-organized systems

bull Multi-Agent Systems (MAS)ndash A system that consists of a number of agents

which interact with one-anotherndash Communication Coordination

Collaboration

23418 48

bull Ant Colony Optimization (ACO)ndash Inspired by ant colony foragingndash pheromone as heuristic information

(stigmergy)ndash Iteration between ConstructAntSolutions and

UpdatePheromones

bull Particle Swarm Optimization (PSO)ndash Inspired by bird flockingndash Heuristic information results from partnersndash Particle Velocity Update

• Collective Intelligence
• Outline
• The Computational Beauty of Nature
• Examples of Collective Behavior in Nature and Society
• Emergence
• Biological motivation Insect Societies
• Insect Societies
• Self Organization
• Stigmergy
• Flocks Herds and Schools
• Boid rules
• Slide 12
• Swarm Intelligence (Contrsquod)
• Components of SI
• Characteristics of SI
• The Web becomes a Giant Brain Some see the Web evolving into a collective brain for humankind
• What is Multi-agent Systems
• What is MAS(Contrsquod)
• Slide 19
• Communication models
• Working together
• Example mechanism Contract Net Protocol (CNP)
• Slide 23
• CNP (Contrsquod)
• Slide 25
• Attributes of Multi-agent Systems
• Applications of MAS
• Applications of MAS (Contrsquod)
• How to simulate SI for search
• Part Ⅱ Ant Colony Optimization (ACO)
• Natural Ants
• Slide 32
• How can the natural ants find the shortest path
• Slide 34
• Problems of AS
• Part Ⅲ Particle Swarm Optimization (PSO)
• Slide 37
• Modeling bird flocking
• Modeling bird flocking (Contrsquod)
• From Bird to Particle
• Features of PSO
• Particle Swarm Optimization Process
• How do particles fly
• Slide 44
• PSO Related issues
• Advantages of PSO
• Summary
• Slide 48

23418 42

Particle Swarm Optimization Process

Step1 Initialize population in hyperspace

Step2 Evaluate fitness of individual particles

Step3 Modify velocities based on previous best and global (or neighborhood) best

Step4 Terminate on some condition

Step5 Go to step 2

23418 43

How do particles fly

bull Combination of gBest and the pBest (lBest)ndash need a compromise

bull lBest can bendash Social the particles around are always the same no matter

where they are in spacendash Geographical the particles around are those whose distance is

the shortest

bull Global PSO vs Local PSOndash the global version converges quickly to a solution but it gets

more easily stuck in local minima

23418 44

Illustrating the velocity update schema of global PSO

23418 45

PSO Related issues

bull Controlling velocities (determining the best value for Vmax)ndash Usually set maximum velocity to dynamic range of variable

bull Usually set c1 and c2 to 2

bull Inertia weight influences tradeoff between global and local explorationndash Good approach is to reduce inertia weight during run (ie from

09 to 04 over 1000 generations)

bull Swarm Size and Neighborhood Size

23418 46

Advantages of PSO

bull Adaptation operates on velocitiesndash Most other methods operate on positionsndash Effect PSO has a builtin momentumndash Particles tend to hurdle past optima ndash an advantage

since the best positions are remembered anyway

bull Simple in conceptbull Easy to implementbull Computationally efficientbull Effective on a variety of problems

23418 47

Summarybull Swarm Intelligence (SI)

ndash an artificial intelligence technique based around the study of collective behavior in decentralized self-organized systems

bull Multi-Agent Systems (MAS)ndash A system that consists of a number of agents

which interact with one-anotherndash Communication Coordination

Collaboration

23418 48

bull Ant Colony Optimization (ACO)ndash Inspired by ant colony foragingndash pheromone as heuristic information

(stigmergy)ndash Iteration between ConstructAntSolutions and

UpdatePheromones

bull Particle Swarm Optimization (PSO)ndash Inspired by bird flockingndash Heuristic information results from partnersndash Particle Velocity Update

• Collective Intelligence
• Outline
• The Computational Beauty of Nature
• Examples of Collective Behavior in Nature and Society
• Emergence
• Biological motivation Insect Societies
• Insect Societies
• Self Organization
• Stigmergy
• Flocks Herds and Schools
• Boid rules
• Slide 12
• Swarm Intelligence (Contrsquod)
• Components of SI
• Characteristics of SI
• The Web becomes a Giant Brain Some see the Web evolving into a collective brain for humankind
• What is Multi-agent Systems
• What is MAS(Contrsquod)
• Slide 19
• Communication models
• Working together
• Example mechanism Contract Net Protocol (CNP)
• Slide 23
• CNP (Contrsquod)
• Slide 25
• Attributes of Multi-agent Systems
• Applications of MAS
• Applications of MAS (Contrsquod)
• How to simulate SI for search
• Part Ⅱ Ant Colony Optimization (ACO)
• Natural Ants
• Slide 32
• How can the natural ants find the shortest path
• Slide 34
• Problems of AS
• Part Ⅲ Particle Swarm Optimization (PSO)
• Slide 37
• Modeling bird flocking
• Modeling bird flocking (Contrsquod)
• From Bird to Particle
• Features of PSO
• Particle Swarm Optimization Process
• How do particles fly
• Slide 44
• PSO Related issues
• Advantages of PSO
• Summary
• Slide 48

23418 43

How do particles fly

bull Combination of gBest and the pBest (lBest)ndash need a compromise

bull lBest can bendash Social the particles around are always the same no matter

where they are in spacendash Geographical the particles around are those whose distance is

the shortest

bull Global PSO vs Local PSOndash the global version converges quickly to a solution but it gets

more easily stuck in local minima

23418 44

Illustrating the velocity update schema of global PSO

23418 45

PSO Related issues

bull Controlling velocities (determining the best value for Vmax)ndash Usually set maximum velocity to dynamic range of variable

bull Usually set c1 and c2 to 2

bull Inertia weight influences tradeoff between global and local explorationndash Good approach is to reduce inertia weight during run (ie from

09 to 04 over 1000 generations)

bull Swarm Size and Neighborhood Size

23418 46

Advantages of PSO

bull Adaptation operates on velocitiesndash Most other methods operate on positionsndash Effect PSO has a builtin momentumndash Particles tend to hurdle past optima ndash an advantage

since the best positions are remembered anyway

bull Simple in conceptbull Easy to implementbull Computationally efficientbull Effective on a variety of problems

23418 47

Summarybull Swarm Intelligence (SI)

ndash an artificial intelligence technique based around the study of collective behavior in decentralized self-organized systems

bull Multi-Agent Systems (MAS)ndash A system that consists of a number of agents

which interact with one-anotherndash Communication Coordination

Collaboration

23418 48

bull Ant Colony Optimization (ACO)ndash Inspired by ant colony foragingndash pheromone as heuristic information

(stigmergy)ndash Iteration between ConstructAntSolutions and

UpdatePheromones

bull Particle Swarm Optimization (PSO)ndash Inspired by bird flockingndash Heuristic information results from partnersndash Particle Velocity Update

• Collective Intelligence
• Outline
• The Computational Beauty of Nature
• Examples of Collective Behavior in Nature and Society
• Emergence
• Biological motivation Insect Societies
• Insect Societies
• Self Organization
• Stigmergy
• Flocks Herds and Schools
• Boid rules
• Slide 12
• Swarm Intelligence (Contrsquod)
• Components of SI
• Characteristics of SI
• The Web becomes a Giant Brain Some see the Web evolving into a collective brain for humankind
• What is Multi-agent Systems
• What is MAS(Contrsquod)
• Slide 19
• Communication models
• Working together
• Example mechanism Contract Net Protocol (CNP)
• Slide 23
• CNP (Contrsquod)
• Slide 25
• Attributes of Multi-agent Systems
• Applications of MAS
• Applications of MAS (Contrsquod)
• How to simulate SI for search
• Part Ⅱ Ant Colony Optimization (ACO)
• Natural Ants
• Slide 32
• How can the natural ants find the shortest path
• Slide 34
• Problems of AS
• Part Ⅲ Particle Swarm Optimization (PSO)
• Slide 37
• Modeling bird flocking
• Modeling bird flocking (Contrsquod)
• From Bird to Particle
• Features of PSO
• Particle Swarm Optimization Process
• How do particles fly
• Slide 44
• PSO Related issues
• Advantages of PSO
• Summary
• Slide 48

23418 44

Illustrating the velocity update schema of global PSO

23418 45

PSO Related issues

bull Controlling velocities (determining the best value for Vmax)ndash Usually set maximum velocity to dynamic range of variable

bull Usually set c1 and c2 to 2

bull Inertia weight influences tradeoff between global and local explorationndash Good approach is to reduce inertia weight during run (ie from

09 to 04 over 1000 generations)

bull Swarm Size and Neighborhood Size

23418 46

Advantages of PSO

bull Adaptation operates on velocitiesndash Most other methods operate on positionsndash Effect PSO has a builtin momentumndash Particles tend to hurdle past optima ndash an advantage

since the best positions are remembered anyway

bull Simple in conceptbull Easy to implementbull Computationally efficientbull Effective on a variety of problems

23418 47

Summarybull Swarm Intelligence (SI)

ndash an artificial intelligence technique based around the study of collective behavior in decentralized self-organized systems

bull Multi-Agent Systems (MAS)ndash A system that consists of a number of agents

which interact with one-anotherndash Communication Coordination

Collaboration

23418 48

bull Ant Colony Optimization (ACO)ndash Inspired by ant colony foragingndash pheromone as heuristic information

(stigmergy)ndash Iteration between ConstructAntSolutions and

UpdatePheromones

bull Particle Swarm Optimization (PSO)ndash Inspired by bird flockingndash Heuristic information results from partnersndash Particle Velocity Update

• Collective Intelligence
• Outline
• The Computational Beauty of Nature
• Examples of Collective Behavior in Nature and Society
• Emergence
• Biological motivation Insect Societies
• Insect Societies
• Self Organization
• Stigmergy
• Flocks Herds and Schools
• Boid rules
• Slide 12
• Swarm Intelligence (Contrsquod)
• Components of SI
• Characteristics of SI
• The Web becomes a Giant Brain Some see the Web evolving into a collective brain for humankind
• What is Multi-agent Systems
• What is MAS(Contrsquod)
• Slide 19
• Communication models
• Working together
• Example mechanism Contract Net Protocol (CNP)
• Slide 23
• CNP (Contrsquod)
• Slide 25
• Attributes of Multi-agent Systems
• Applications of MAS
• Applications of MAS (Contrsquod)
• How to simulate SI for search
• Part Ⅱ Ant Colony Optimization (ACO)
• Natural Ants
• Slide 32
• How can the natural ants find the shortest path
• Slide 34
• Problems of AS
• Part Ⅲ Particle Swarm Optimization (PSO)
• Slide 37
• Modeling bird flocking
• Modeling bird flocking (Contrsquod)
• From Bird to Particle
• Features of PSO
• Particle Swarm Optimization Process
• How do particles fly
• Slide 44
• PSO Related issues
• Advantages of PSO
• Summary
• Slide 48

23418 45

PSO Related issues

bull Controlling velocities (determining the best value for Vmax)ndash Usually set maximum velocity to dynamic range of variable

bull Usually set c1 and c2 to 2

bull Inertia weight influences tradeoff between global and local explorationndash Good approach is to reduce inertia weight during run (ie from

09 to 04 over 1000 generations)

bull Swarm Size and Neighborhood Size

23418 46

Advantages of PSO

bull Adaptation operates on velocitiesndash Most other methods operate on positionsndash Effect PSO has a builtin momentumndash Particles tend to hurdle past optima ndash an advantage

since the best positions are remembered anyway

bull Simple in conceptbull Easy to implementbull Computationally efficientbull Effective on a variety of problems

23418 47

Summarybull Swarm Intelligence (SI)

ndash an artificial intelligence technique based around the study of collective behavior in decentralized self-organized systems

bull Multi-Agent Systems (MAS)ndash A system that consists of a number of agents

which interact with one-anotherndash Communication Coordination

Collaboration

23418 48

bull Ant Colony Optimization (ACO)ndash Inspired by ant colony foragingndash pheromone as heuristic information

(stigmergy)ndash Iteration between ConstructAntSolutions and

UpdatePheromones

bull Particle Swarm Optimization (PSO)ndash Inspired by bird flockingndash Heuristic information results from partnersndash Particle Velocity Update

• Collective Intelligence
• Outline
• The Computational Beauty of Nature
• Examples of Collective Behavior in Nature and Society
• Emergence
• Biological motivation Insect Societies
• Insect Societies
• Self Organization
• Stigmergy
• Flocks Herds and Schools
• Boid rules
• Slide 12
• Swarm Intelligence (Contrsquod)
• Components of SI
• Characteristics of SI
• The Web becomes a Giant Brain Some see the Web evolving into a collective brain for humankind
• What is Multi-agent Systems
• What is MAS(Contrsquod)
• Slide 19
• Communication models
• Working together
• Example mechanism Contract Net Protocol (CNP)
• Slide 23
• CNP (Contrsquod)
• Slide 25
• Attributes of Multi-agent Systems
• Applications of MAS
• Applications of MAS (Contrsquod)
• How to simulate SI for search
• Part Ⅱ Ant Colony Optimization (ACO)
• Natural Ants
• Slide 32
• How can the natural ants find the shortest path
• Slide 34
• Problems of AS
• Part Ⅲ Particle Swarm Optimization (PSO)
• Slide 37
• Modeling bird flocking
• Modeling bird flocking (Contrsquod)
• From Bird to Particle
• Features of PSO
• Particle Swarm Optimization Process
• How do particles fly
• Slide 44
• PSO Related issues
• Advantages of PSO
• Summary
• Slide 48

23418 46

Advantages of PSO

bull Adaptation operates on velocitiesndash Most other methods operate on positionsndash Effect PSO has a builtin momentumndash Particles tend to hurdle past optima ndash an advantage

since the best positions are remembered anyway

bull Simple in conceptbull Easy to implementbull Computationally efficientbull Effective on a variety of problems

23418 47

Summarybull Swarm Intelligence (SI)

ndash an artificial intelligence technique based around the study of collective behavior in decentralized self-organized systems

bull Multi-Agent Systems (MAS)ndash A system that consists of a number of agents

which interact with one-anotherndash Communication Coordination

Collaboration

23418 48

bull Ant Colony Optimization (ACO)ndash Inspired by ant colony foragingndash pheromone as heuristic information

(stigmergy)ndash Iteration between ConstructAntSolutions and

UpdatePheromones

bull Particle Swarm Optimization (PSO)ndash Inspired by bird flockingndash Heuristic information results from partnersndash Particle Velocity Update

• Collective Intelligence
• Outline
• The Computational Beauty of Nature
• Examples of Collective Behavior in Nature and Society
• Emergence
• Biological motivation Insect Societies
• Insect Societies
• Self Organization
• Stigmergy
• Flocks Herds and Schools
• Boid rules
• Slide 12
• Swarm Intelligence (Contrsquod)
• Components of SI
• Characteristics of SI
• The Web becomes a Giant Brain Some see the Web evolving into a collective brain for humankind
• What is Multi-agent Systems
• What is MAS(Contrsquod)
• Slide 19
• Communication models
• Working together
• Example mechanism Contract Net Protocol (CNP)
• Slide 23
• CNP (Contrsquod)
• Slide 25
• Attributes of Multi-agent Systems
• Applications of MAS
• Applications of MAS (Contrsquod)
• How to simulate SI for search
• Part Ⅱ Ant Colony Optimization (ACO)
• Natural Ants
• Slide 32
• How can the natural ants find the shortest path
• Slide 34
• Problems of AS
• Part Ⅲ Particle Swarm Optimization (PSO)
• Slide 37
• Modeling bird flocking
• Modeling bird flocking (Contrsquod)
• From Bird to Particle
• Features of PSO
• Particle Swarm Optimization Process
• How do particles fly
• Slide 44
• PSO Related issues
• Advantages of PSO
• Summary
• Slide 48

23418 47

Summarybull Swarm Intelligence (SI)

ndash an artificial intelligence technique based around the study of collective behavior in decentralized self-organized systems

bull Multi-Agent Systems (MAS)ndash A system that consists of a number of agents

which interact with one-anotherndash Communication Coordination

Collaboration

23418 48

bull Ant Colony Optimization (ACO)ndash Inspired by ant colony foragingndash pheromone as heuristic information

(stigmergy)ndash Iteration between ConstructAntSolutions and

UpdatePheromones

bull Particle Swarm Optimization (PSO)ndash Inspired by bird flockingndash Heuristic information results from partnersndash Particle Velocity Update

• Collective Intelligence
• Outline
• The Computational Beauty of Nature
• Examples of Collective Behavior in Nature and Society
• Emergence
• Biological motivation Insect Societies
• Insect Societies
• Self Organization
• Stigmergy
• Flocks Herds and Schools
• Boid rules
• Slide 12
• Swarm Intelligence (Contrsquod)
• Components of SI
• Characteristics of SI
• The Web becomes a Giant Brain Some see the Web evolving into a collective brain for humankind
• What is Multi-agent Systems
• What is MAS(Contrsquod)
• Slide 19
• Communication models
• Working together
• Example mechanism Contract Net Protocol (CNP)
• Slide 23
• CNP (Contrsquod)
• Slide 25
• Attributes of Multi-agent Systems
• Applications of MAS
• Applications of MAS (Contrsquod)
• How to simulate SI for search
• Part Ⅱ Ant Colony Optimization (ACO)
• Natural Ants
• Slide 32
• How can the natural ants find the shortest path
• Slide 34
• Problems of AS
• Part Ⅲ Particle Swarm Optimization (PSO)
• Slide 37
• Modeling bird flocking
• Modeling bird flocking (Contrsquod)
• From Bird to Particle
• Features of PSO
• Particle Swarm Optimization Process
• How do particles fly
• Slide 44
• PSO Related issues
• Advantages of PSO
• Summary
• Slide 48

23418 48

bull Ant Colony Optimization (ACO)ndash Inspired by ant colony foragingndash pheromone as heuristic information

(stigmergy)ndash Iteration between ConstructAntSolutions and

UpdatePheromones

bull Particle Swarm Optimization (PSO)ndash Inspired by bird flockingndash Heuristic information results from partnersndash Particle Velocity Update

• Collective Intelligence
• Outline
• The Computational Beauty of Nature
• Examples of Collective Behavior in Nature and Society
• Emergence
• Biological motivation Insect Societies
• Insect Societies
• Self Organization
• Stigmergy
• Flocks Herds and Schools
• Boid rules
• Slide 12
• Swarm Intelligence (Contrsquod)
• Components of SI
• Characteristics of SI
• The Web becomes a Giant Brain Some see the Web evolving into a collective brain for humankind
• What is Multi-agent Systems
• What is MAS(Contrsquod)
• Slide 19
• Communication models
• Working together
• Example mechanism Contract Net Protocol (CNP)
• Slide 23
• CNP (Contrsquod)
• Slide 25
• Attributes of Multi-agent Systems
• Applications of MAS
• Applications of MAS (Contrsquod)
• How to simulate SI for search
• Part Ⅱ Ant Colony Optimization (ACO)
• Natural Ants
• Slide 32
• How can the natural ants find the shortest path
• Slide 34
• Problems of AS
• Part Ⅲ Particle Swarm Optimization (PSO)
• Slide 37
• Modeling bird flocking
• Modeling bird flocking (Contrsquod)
• From Bird to Particle
• Features of PSO
• Particle Swarm Optimization Process
• How do particles fly
• Slide 44
• PSO Related issues
• Advantages of PSO
• Summary
• Slide 48