La simulation agent et des applications

Cour d’introduction général

Juliette Rouchier, GREQAM- CNRS, 2 rue de la Charité, 13236 Marseille Cedex 02, France

rouchier@ehess.univ-mrs.fr

SMA: AEIO model agents

Real or virtual entity, autonomous, local percreption of environment, capable of acting. Shared perception, image of self and others, memory, goals, beliefs.

environnement Objects : caracteristics, dynamic evolution lawsFormalised by a grid (automata) or a network

interactionscommunication groups, language, communication protocols (ie : offer

competencies, answer)Interpretation of messages

organisationCorrelation concerning the evolution of certain entities, temporal

organisation Groups or predefined networks with tasks, shared norms, links for

communication

Complex system (Simon, 61)

Self-organisation, emergence : what wasn’t defined in the individual entities’ behaviour

Traffic jam, adaptation,...

System limits interacting entitiesdynamics controlfeedback

OBSERVED System Points of view

expectations emergence

Artificial: human made, with a goal, imitating, imperative

Black box of a system

Cellular automata

Automata S = set of states I = set of inputs O = set of outputs Transition function

State OutputInputs

Network of automata with special architecture : inputs of some are outputs for others

Intelligence artificielleSystème expert et multi-expertsLogique formelleThéorie de l’information

Cybernétique - contrôleMinskyBateson

Example : Game of life

MANTA : interactions through a resource and specialisation / learning through simple feedbacks

Reactive agents Ants, egg, larva, cocoon

Don’t perceive the others, but stimuli in the environmentCan choose an action, according to activity levels (thresholds)Competing tasks: cure, feed, carry gather food

A. Drogoul

Goal: To represent labour division with very simple agents

Inspiration: ants societies

Build a framework that is useful to computer scientists and ethologists

MANTA : interactions through a resource and specialisation / learning through simple feedbacks

A. Drogoul

• Success of sociogenesis in about 20% cases, without any centralised decision• Collaboration without any knowledge of the others (intelligence of the programmer)• Specialisation without loosing adaptativity when thresholds of reactions are well adapted• Organisation more or less complex and division of labour (different age > different tasks)• Progressive complexification > diverse learning processes – reinforcement engendered by competition

Emergence of hierarchies (Doran, Palmer) Goal: to produce a hierarchical society from an egalitarian

society Hypothesis: the resource characteristic is the explanation

Resources requires many hunters (complexity)defined by: location, instances, energy, type, complexity,

distance for agents to get it, sime be4 renewal

Agents seek to “stay alive” (energy)cognitive and action “if then” rules (1 action per time-

step)memory: resource model, social model, message buffer,

miscellanous (hunger, behaviour mode, perception range)location, speed, sensory range, skill, energy, hunger limitmodes: autonomous, recruiting, biding, executing

Recruitment process: leaders and followers an agent calls up -> the other proposes a bid > accepts the bid >>

groupwhen leader, the agent involves its own group when accepting

(groups get to be parts of groups)Simulation Variables:

recruitment rules, Fidelity (how long it stays in the group after activity stops) conditions of agreement to follow the leader

Observation indicators:depth of hierarchy (individual and global)

Results:Groups appear and last, autonomous actions, if resources are gathered in a place, groups migrate in this area Need of low complexity resource to have step by step hierarchy buildingDecrease of productivity with rigid social structures (fidelity).

Emergence of hierarchies (Doran, Palmer)

Growing artificial societies. Social science from the bottom-up (sugarscape)

Environment – resources Agents : layers building

Needs / satisfaction / perception / movement > migrations, differenciations Reproduction / death > s₫lection Inheritage > less inequalities but less selection

Culture : gene dissociating two groups Fight for access to resource > elimination or assimilation

Exchanges : two resources and different needs With exchanges > reduction of mortality and increasing of inequalities

Economic hypotheses test Equilibrium appear

Pollution, preference evolution, disease transmission in migration

Agents

No learning

Learning

Reactive behavior Cognitive behavior

Perception

Action

Perception

Decision

ActionPerception

Action

Assessment

PerceptionDecisionAction

Assessment

NOT INDEPENDENT !!!

Autonomy vs independenceSeparation agent - environment / ability to adapt in evolving environment

Actions ordering, interpretation, choice, behavioural change

Directe Communication : give informations / distribute tasks / solve conflicts / learn

Representation of others : how, with whom to communicate, what knowledgeRepresentation of the relationship: familiarity, trust Type of language, interpretation of a message

Interactions

Direct Indirect

Modification of representations / beliefsModification of goals

Indirecte Communication: évolve without consciousness of others’ presence but react to the transformation of environment

signals left in environment (stimuli - externalities)

COGNITIVE

REACTIVE

Memory

No memory

Conservation of thresholds

Conservation of messages received

Conservation of messages received and sent

Conservation of large amount of information about the context of the messages

Organisation

Built in organisational elements Networks

who to communicate withdelegationcommitmentdependence – authorityknowledge of abilities

Repartition of tasks – roles - abilitiesTemporal organisation of the systemWho has access to ressource / control overNorms of behaviour that exist in the system – interpretation for agents

Emerging patterns that retroact on the system by constraining agentsCompetitionEmerging norms (regularities of behaviour in the group)

Diverse representations of social behaviour From goals to intentions (commitment)

Blinded : Until the agent believes it has accomplished its intention Single-minded : As long as it thinks it is possible Open minded : As soon as it has the goal

Some strategies for negotiation

Always concede Be competitive Be cooperative : look for a mutually acceptable solution Inaction Break

Reactive agents often coordinate through the environment Two approaches for cognitive agents

define mutual beliefs, joint desires and joint intentions define norms and conventions.

Methods to have agents evolve Reinforcement learning

Utility function > evaluate results and classify them Learning / memory capacity / Change of behavior rules

Comparison and copy of others’ methods Information diffusion or behaviour diffusion Choice of relevant agents to copy (trust, network) Mecanisms to adopt behaviours

Genetic algorithms (population level – social learning) « fitness » function, reproduction, meeting, mutation

Memetics� Inspiration of genetics in building of representation

the noosphère (Morin), « memes » (Dawkins), epidemiology of representations (Sperber).

� Hales (following Bura): « Memes » are on animats using an environment and subject to selection.

Each meme : propensity to mute and to reproduce; fights with neighbors and strenghthening.

3 stages : calculation of satisfaction, mutation, replication.

If satisfied increased aggressiveness and decreased mutation If not satisfied the reverse is trueExistence of a meta-meme: open-mindedness that suppresses that phenomena

Results Scenario : Just enough food, too much food, predators.

Stabilisation of size of animat population able to occupy an area (carrying capacity).

killing memes population can grow but is a sign of instability in the system

open-mindedness meme help global survival of the population

En ce qui concerne les apprentissages individuels, Bourgine [1993] distingue plusieurs niveaux de rationalité des agents selon leur relation à leur environnement et leur capacité à modéliser le réel.

•Les agents réactifs réagissent de manière fixe à l’information provenant de leur environnement, sur le mode stimulus-réponse (réponse sensori-motrice ou " pavlovienne " héritée génétiquement) : il y a absence d’apprentissage. •Les agents hédoniques apprennent (par auto-renforcement) à modifier leur comportement afin d’augmenter leur " plaisir ". Ils sont capables d’anticipations " hédoniques " et d’adaptation lente à partir de leur expérience historique, ce qui suppose un niveau de conscience plus élevé que l’agent réactif (consciousness). •Les agents éductifs sont dotés d’une capacité de modélisation de leur environnement, ce qui suppose la capacité de former des représentations symboliques, de simuler les conséquences d’une action sur leur environnement, et donc un niveau de conscience plus élevé (awareness).

Selon une perspective plus proche des catégories de l’économiste, Walliser [1997] propose une typologie des processus qui permettent de converger vers un équilibre en théorie des jeux. Il en distingue quatre, soit par ordre décroissant des capacités cognitives attribuées aux agents :

•Dans un processus EDUCTIF, chaque joueur dispose d’assez d’information pour simuler parfaitement le comportement des autres joueurs, ce qui conduit immédiatement à l’équilibre : il n’y a pas d’apprentissage.•Dans un apprentissage EPISTEMIQUE, chaque joueur révise ses croyances relatives aux stratégies des autres adversaires à partir des informations qu’il a pu observer (Fudenberg, Levine [1998]).•Dans un apprentissage COMPORTEMENTAL, chaque joueur modifie sa stratégie compte tenu des résultats observés de ses propres actions dans le passé (agent hédonique).•Dans un apprentissage ÉVOLUTIONNAIRE, chaque joueur joue une stratégie fixe qui se reproduit proportionnellement au gain obtenu lors de confrontations aléatoires (agent réactif).

Simulation : temporal evolution

Initial setting Environment, number of agents, dotations, ...

A time-stepenvironment evolves

agents perceiveagents make choice

agents actagents communicate

Final setting

Model (agents, environment, interactions, organisation)Parameters : number of agents, learning principle, costs

Simuler = chercher les « causes » de l’auto-organisation

Observer l’auto-organisation : critère global, critère individuel, aggrégation de données individuels, liens / rencontre des agents, représentations individuelles and collectives.

INDICATEURS PERTINENTS POUR DECRIRE DES PHENOMENES «EMERGENTS »

Analyse est possible à travers la comparaison (= « sensibilité aux changement des valeurs de paramètres ») : changement de la situation initiale ou des règles de l’univers – observation des résultats finaux et des processus intermédiaires

CHERCHER LES IMPLICATIONS DES REGLES ET DE LA SUCCESSION D’EVENEMENTS