topic 8: coordination and control applications coordination and control applications a reference...

Topic 8: Coordination andControl Applications

coordination and control applications a reference architecture ant-based coordination and control

Overview

Coordination and Control applications characteristics (2-levels, task-oriented, large scale, dynamic, requirements) software architecture decentralization

Manufacturing Control requirements

functional non-functional flexibility under disturbances and change

software architecture basic software architecture

task agents mobile units / tasks

resource agents resources environment graph

constraints: product recipes coordination

coordination through delegate MASs exploit the environment feasibility delegate MASs + appl. characteristics exploration delegate MASs intention delegate MASs

Multi-Agent Systems forCoordination and Control Applications

family of applications, characterized by control application

underlying (physical or software) system that needs to be controlled resources – static entities mobile entities

on top: software system to “control” the underlying system different order of magnitude of evolution speed

task-oriented application domain a task entails moving through the environment (mobile entities) and

performing operations using resources (static entities) large / huge scale

number of entities physical distribution

very dynamic nature resources / connections / tasks

complex functional / non-functional requirements coordination ! cooperation ! plan ahead !!

Multi-Agent Systems forCoordination and Control Applications

examples manufacturing control AGV-based warehouse management traffic control web service coordination

control application underlying (physical or software) system that needs to be controlled

resources – static entities mobile entities

on top: software system to “control” the underlying system different order of magnitude of evolution speed

task-oriented application domain a task entails moving through the environment (mobile entities) and performing operations using resources (static

entities) large / huge scale

number of entities physical distribution

very dynamic nature resources / connections / tasks

complex functional / non-functional requirements coordination ! cooperation ! plan ahead !!

Manufacturing control

generic / simplified perspective the underlying system:

manufactory for building products

resources – static entities machines (painting, milling, drilling, …) conveyor belts ..

environment: graph–like structure dynamic directed graph

nodes: resources edges: connections

embeds mobile entities

mobile entities – tasks / orders intermediate products travel over resources

Manufacturing Control Requirements

Functional Production of ordered products Coordination of resources and orders

Adaptive: Changes and disturbances (all-the-time)

Quality of service / throughput …

• Disturbances– machine break-down– missing tools– missing materials– etc.

• Changes– new technology– new markets– new organisational structures– etc.

OUT

IN

= resource / machine = connection

OUT

IN

= resource / machine = connection

Type-1

Type-1

Type-1

Type-1

Type-2

Type-2

Type-2

Type-2

Type-3

Type-3

Type-3

Type-3

Type-4

Type-4

Type-5

OUT

IN

= resource / machine = connection

OUT

= resource / machine = connection

order-1

OUT

= resource / machine = connection

order-1

order-2

OUT

= resource / machine = connection

order-1

order-2

order-3order-4

order-5order-6

OUT

= resource / machine = connection

Multi-Agent Systems forCoordination and Control Applications

Centralized approaches consider the problem to be an optimisation problem operations research / static and dynamic feasibility ...?

…

Multi-Agent Systems forCoordination and Control Applications

…

Distributed approaches local decision makers, which cooperate / coordinate...

vehicles carrying orders manufactory network - resources

decentralized architectures hold several advantages inherently adaptive Scalable? / local observation and control …

crucial problem remains: deal with scale and complexity

Solution? Distributed approaches

…

compromises ... hierarchical models

- e.g. based on geographical characteristics... compromises on flexibility, performance, complexity e.g. 2-level distribution... [Klaus Fischer ’95]

pure decentralization simple local rules + rely on emergence compromise on optimality [Tamas Mahr ’08]

What is at the heart of the problem...

local decision makers require global information for adequate decision making

What could be at the heart of the solution...

local decision makers find/isolate only that global information

that is directly relevant for adequate decision making

Multi-Agent Systems forCoordination and Control Applications

…

Reference model for Coordination and Control applications

Decentralized components / agents Environment-centric coordination model

Ant-inspired…

Basic Components of the Reference Architecture

based on PROSA reference architecture

Agent types Resource agent Task agent

Environment

Basic Components of the Reference Architecture (cont.)

Environment software environment reflects physical environment

dynamic graph nodes: resources edges: connections

support interaction / interface

with resources embeds virtual mobile entities

Basic Components of the Reference Architecture (cont.)

Resource agent Reflects a resource in the manufacturing system Autonomous for optimizing its own behaviour

schedule lifetime …

Local topology Entries and exits of the resource Exits connected to its entries and vice versa

Services Capabilities() >> resource capabilities representation Status() >> availability representation PossibleOps(setOfOps) >> possible processing steps Perform(…) >> execute an operation Set(…) >> e.g. download or select an NC

program What-if / predictive model (see further)

Basic Components of the Reference Architecture (cont.)

Task agent overall “goal”:

fulfill task by moving over resources in some

correct sequence fulfilling expected timing and quality requirements

of the order

Product recipe Reflects a product type for tasks, not product instances Knows possible ways to correctly produce instance(s) and provides this

information to … Minimizes assumptions about resource availability

E.G. Support 3 and 5 axis alternatives Methods

InitialState() >> product instance state representation

PossibleNextSteps(currentState) >> set of operations NextState(currentState, operation, result) >> state rep

Basic Components of the Reference Architecture (cont.)

Task agent overall “goal”:

fulfill task by moving over resources in some

correct sequence fulfilling expected timing and quality requirements

of the order

BDI – Beliefs–Desires–Intentions beliefs

resources possible (feasible) paths for reaching resources other task agents ?

desires / options several paths through the infrastructure / resources

intention a chosen path

…

Coordination model Basic entities in place

environment infrastructure agents vehicle agents

Now the system should support agents fulfilling tasks tasks are trips through the manufactory, along correct sequence of

resources

...taking timing and quality requirements into account... minimize production time avoid traffic jams ...

all this in an environment that changes constantly …and in which task agents enter the system constantly …

Task agents – BDI ?

how ? first alternative: task agent responsible for gathering, reasoning upon

and distributing information about resources

capabilities / quality / … topology expected schedule

about paths: find out feasible routes (match routes against product recipe) contact resources on paths judge on the quality of these paths

about intentions: communicate own intentions to other agents negotiate with other agents to align all agents’ intentions reserve resources if suitable

all this in an environment that changes constantly …and in which task agents enter the system constantly …

complex

Task agents – BDI ?

how ? second alternative:exploit environment to relief task agents

…

delegate MAS

have simple, small-scale agents (ants) roam environment and enrich enviroment with valuable information

optional paths intentions

align intention with intentions of other task agents through resource agents through refresh

Delegate MAS:Ant-based Coordination and Control

three kinds of delegate MASs Feasibility ants Exploration ants Intention ants

1. Ant agents

2. Pheromone deposition spaces

attached to each resource/enty/exit

! evaporation and refresh

Delegate MAS-1: Feasibility

Feasibility ants Purpose

maintain “road signs” (adapted on change in environment) road sign indicates possible sequences of operations

e.g.“if you go from this node to node N1, you are able to move over resources to perform operations op1 – op2 – (op3 – op4)* – op5”

Sent out by resources (esp. out resources) independent of orders / current load / …

Constrain routing to legal paths Ensure that the process plan can be executed properly Leave resource allocation choices open when there is no technical ground to

select or forbid Other feasibility concerns exist

Deadlock, livelock, starvation Quality and reliability issues

OUT

IN

= resource / machine = connection

Delegate MAS-1: Feasibility

Feasibility ants (continued) Information usage

Task agent, moving downstream, retrieves latest information when investigating routing options

Task agent uses this information to inspect its recipe

Delegate MAS-2: Exploration

Exploration ants Task agents “delegate” the exploration of possible trajectories

create exploration ants at a regular frequency that perform Forward exploration from current position / present Backward exploration from due date / shipping point(s)

Exploration ant virtually executes a possible scenario on behalf of the task agent and reports back the route and performance estimates

Enforces feasibility constraints Uses/reflects the decision module of the task agent Cloning and other variants are possible

OUT

order-1

Tries to explore feasible solutions time (travel + queuing + processing) and quality if task agent would follow this path ?

Searching for solutions is guided by local pheromones Reports result of solution to the corresponding Task Agent

Ra

Rb

Rc

Rd

Ra

Rb

Rc

Rd

Ra

Rb

Rc

Rd

Ra

Rb

Rc

Rd

Ra

Rb

Rc

Rd

resource schedule

+ what-if scenarios

orders

time

??

now

resource schedule

if reserved later on

orders

timenow

Delegate MAS-2: Exploration

Exploration ants (information usage) Task agent keeps a set of candidate routes

Performance criteria (good performance) Complementarity (reduce vulnerability)

Candidate routes are refreshed regularly Explicit/deterministic or stochastically Old candidates evaporate

Delegate MAS-3: Intention

Intention ants and forecasting

Task agent picks one candidate: intention Refresh before selection/replacing Warm-up phase

OUT

order-1

Task agent creates intention ants at a regular frequency to virtually execute the route of the current intention of their task agent

Intention ant informs visited resources on its virtual journey about the task agent's intention Makes a reservation/booking

Resource agents adapt what-if self-model Bookings evaporate if not refreshed

Ra

Rb

Rc

Rd

Delegate MAS –Intention reconsiderations

Frequency of exploration? Strategy to decide upon changing intentions

“Socially acceptable behaviour” Too nervous >> unreliable forecasting Too calm >> no adaptation to change/disturbances Too calm >> lock-in in early explorations

No change for small gains, limits on frequent changing

Less nervous for near future choices

Probabilistic changing and high enough refresh rate!

Experience

manufacturing control testbed / simulation environment considerable advantage w.r.t. adaptibility to dynamic environment

Experience

… traffic control?

45

Prototype: ExperimentsPrototype: Experiments

2

3

1

Always shortest route

Minimize travel time

Traffic DistributionAvg. Travel Time

Always avoid traffic jams

Conclusion

Delegate MAS reference model core abstractions

environment task/vehicle agents - basically BDI resource/infrastructure agents

coordination model environment centric light-weight ‘ants’ + pheromones bring relevant global information to local task agents

spread relevant global information through environment cope with dynamics

...

Conclusion: Delegate MAS Feasability delegate MAS

distribute “beliefs” about feasible paths drop the beliefs in the environment

task agents do not need to gather and maintain these beliefs

Exploration delegate MAS use the environment to find out the quality of different options

task agents do not need to directly contact and negotiate with resources

Intention delegate MAS use the environment to propagate intentions through the environment

task agents do not need to maintain beliefs and reason uponthe intentions of other agents’ for coordinating over resources

reduced complexity of task agent architecture

Many challenges / Open Issues There’s a cost …

additional infrastructure open resources computational/communication cost

needs to be managed properly! suitable refresh rate, cloning budgets, hop limits

Emergent behaviour / qualities … ? purely selfish agents – sufficient for overall optimization ?? homogeneous or heterogeneous?

Many parameters tune ? adaptive strategy ?

Coordination between resources BDI architecture Resource agent architecture …