Software Agents and Applications

Maria Ganzha & Marcin Paprzycki

Systems Research Institute

Polish Academy of Science

GeneralitiesGeneralities

1. Agent Metaphors2. Definitions 3. Critical Analysis

What is an Agent? Seem to be as many definitions as their users Dictionary:

a·gent n.– One that acts or has the power or authority to act– One empowered to act for or represent another: an

author's agent; an insurance agent– A means by which something is done or caused;

instrument– A force or substance that causes a change: a chemical

agent; an infectious agent– A representative or official of a government or

administrative department of a government: an FBI agent– A spy– Linguistics. The noun or noun phrase that specifies the

person through whom or the means by which an action is effected

Agent Technology Metaphors (1) Based on human agents

travel agent insurance / real-estate secret agent personal assistant (a.k.a. secretary)

Result agentshave specialized knowledgerepresent our interests find / filter / customize information

P. Maes, Agents that Reduce Work and Information Overload, CACM, 37, 1994, 31-40

Agent Technology Metaphors (2) Agents as “effect” of software evolution

OOactive objectsagents

Result agentsmethodology for software creation

complex systems build out of interacting intelligent agents

? using agent environments ?

N. R. Jennings, An agent-based approach for building complex software systems, CACM, 44 (4), 2001, 35-41

Agent Technology Metaphors (3) Other agents

Bio-agents (J. Liu, 2001) based on cellular automata / “ants” simple behavior (no intelligence) “intelligence” emergent behavior

Embodied agents Agent-robots (robo-soccer)

Agents in modeling and simulations modeling social interactions (H. Hexmoor, 2003)

modeling crowd behavior (S. Bandini, 2005)

Agent Definition (1) Definition 1

Anything that can be viewed as perceiving its environment through sensors and acting upon that environment through effectors (Russell and Norvig, 1995)

Definition 2Software entities that carry out some set of

operations on behalf of a user or another program with some degree of independence or autonomy, and in so doing, employ some knowledge or representation of the user’s goals or desires (IBM, 1997)

Agent Definition (2)

Definition 3Autonomous system situated in a dynamical

environment acting independently of its restrictions and fulfilling in it a set of goals or directives for which it was created (Maes, 1998)

Definition 4An encapsulated computer system that is

situated in some environment and that is capable of flexible action in that environment in order to meet its design objectives (Jennings et al, 2000)

there is more (or no definition at all; e.g. Liu)…

Agent Definition (3)• Features of software

agents– reactiveness– ability to

communicate– capacity for

cooperation– reasoning based on

collected knowledge– capacity for

reasoning– intelligence– adaptability

– interactiveness– learning ability– proactiveness– goal orientation– friendliness and

reliability– mobility– autonomy– situatedness– …

Agent Definition (4) No agreed-on definition No agreed-on typology Researchers take various characteristics from

the set proposed for agents and make their own “definition” (Galant & Tyburcy, 2001)

Let us use basic intuitionsSoftware artifact

autonomous represents user and has delegation to act on her

behalf interacts with environment and other agents possibly mobile

Client-Server vs. Mobile Agents

Client

Server Server

Client

Agent

Server

Client

Agent

Server

Client

Agent

Traditional Mobile Agent-Based

Mobile agent agent that:can move from one computer to anotheruser-directed / autonomous / mixed

Examples of Agent Systems

• Internet– avatars– search/browsing– monitoring– info-agents

• e-mail• meeting assistants• news

– e-commerce

• Complex systems– trading

• electricity• stock market

– network and resource management

– production control– supply chain

management– complex system

simulations

Why Agents? (some reasons)

Operate where data and/or resources are combine data from multiple repositories possible to combine with GRID computing

load balancing quality of service / service level agreement

Disconnected operations and autonomy short “on-line” times (ideal for mobile devices)

Redundancy / fail-safe behavior multiple agents can “back-up” each other

Few General Criticisms (there is more)

Everything that can be done with agents can be done some other way…

Have been around for years and… ??? Security have to protect

host(s) from agent(s) / agent(s) from host(s)agent(s) from agent(s) / communication

Only one step removed from a virus In many cases agents were not necessary! WHAT, NO DEFINITION?

Possible Responses Agents as a unified framework

Security and virus-related concerns can be addressed (can it be worse than Windows™?)

Agent systems are (and depend on) “bleeding edge” technology that “disappears” (Aglets, BlueStone, e-speak,…)

No way to establish usability / worthlessness until full blown implementations tried…

Specific Criticisms Nwana, Ndumu (1999) A perspective on software

agents research, Knowledge Engineering Review, 14 (2), 1–18

Information discovery problem where is the relevant data and how to keep up with the dynamics of

the Internet? Communication problem

how to make different systems (including agent-agent and agent-non-agent) to communicate with each other?

Legacy software problem how to make agents interact with legacy systems?

Reasoning and coordination problem how to reason about the retrieved data (“AI”)?

Ontology problem how to make different systems understand each other?

8-years Later (1) Substantial progress

information discovery web / text mining / filtering

communication ACL + FIPA SL (and other) standards by FIPA

legacy systems Java – wrappers

reasoning and coordination a lot of progress in various areas of AI

8-years Later (2) Ontology problem

hot research area SemanticWeb supported by EU

need of standards is being addressed need of “demarcated” web-pages / sites

critical mass of available data required to establish for each proposed approach its

feasibility strengths weaknesses

agents proposed as a natural technology for the SemanticWeb (Hendler, 2001)

GeneralitiesGeneralities

4. FIPA – towards agent environments

Foundation for Intelligent Physical Agents (FIPA)Standards Organization

http://www.fipa.org/

Agent Platform (AP) infrastructure for agent environment key agents for management

Agent Communication Language (ACL)communicative acts

Contents of messages (FIPA SL) Protocols – later

message patterns transport level

FIPA reference architecture Specifies rules for agent societies

to interoperate and to be managed ACL formally defined

syntax and semantics Utilized in modern agent environments

in a long-run may provide interoperability

Other AgentsSpeaks IIOP

or RMI

Message Transport System

AgentManagement

System

DirectoryFacilitator

Agent Platform

Agent1

YellowPages

Supervisorycontrol

Authentication of agents

Registration of agents

Agentn

Agent Communication Language Semantically defined standard vocabulary and syntax

string-based messages An ACL Message:

Structured message, targeted for flexible communication Performative (INFORM, QUERY, REFUSE, …) Addressing (to/from) ConversationID – Used to link messages in same

conversation In reply to – Sender uses to help distinguish answers Reply with – Another field to help distinguish answers Reply by – Used to set a time limit on an answer Language – Specifies which language is used in the

content Ontology – Specifies which ontology is used in the content Protocol – Specifies the protocol Content – This is the main content of the message

Why Use Agent Environments? (1) No need to implement basic services

agent “platform” agent “containers” management infrastructure (such as)

Agent Management System (AMS) – white pages Directory Facilitator (DF) – yellow pages

No need to implement agent-management ontology and functionalities agent is registered within its constructor

it is given a name and an address

agent class provides a simplified interface to access the services of the DF (registration, searching, …)

Why Use Agent Environments? (2) No need to implement Message Transport and Parsing

automatically (and possibly efficiently) done by the environment when sending/receiving messages

Interaction Protocols must only be extended via handle methods

In modern environments FIPA standards are immediately supported… (to a certain extent)

SUMMARIZING some form of an infrastructure to support agent system

development is provided by the environment one can concentrate on designing and implementing

agents systems (high level functionalities)

GeneralitiesGeneralities

5. Everything You Always Wanted to Know About (JADE) Agents But Were Afraid to Ask

/almost Woody Allen/

JADE (Java Agent DEvelopment Framework) http://jade.tilab.com target users: agent programmers for MAS includes

a library of classes to create agents a distributed runtime environment (an agent platform)

that provides agent services

life-cycle, white-page, yellow-page, message transport, message encoding

set of tools to support the debugging phase remote monitoring agent, dummy agent, sniffer agent

designed to support scalability from debugging to deployment from small scale to large scale to micro scale

claims to comply with FIPA specifications

JADE

Distributed architecture of a JADE Agent Platform

Network protocol stack using RMI or IIOP

JRE 1.5 JRE 1.5 JRE 1.5

Jade Main-container Jade Agent Container Jade Agent Container

Jade distributed Agent Platform

DF

Age

nt

AM

S A

gent

App

licat

ion

Age

nt

App

licat

ion

Age

nt

App

licat

ion

Age

nt

App

licat

ion

Age

nt

App

licat

ion

Age

nt

App

licat

ion

Age

nt

Host 1 Host 2 Host 3

App

licat

ion

Age

nt

App

licat

ion

Age

nt

Internal architecture of a generic JADE agent

JADE library of interaction protocols and of generic agent behaviors

Agent and ACL messages

controls its thread of executionhas a private proxy of the life-

cycle manager decides when to read

messages and which messages to read the transport mechanism fills a

private queue but it does not call the agent code (no automatic callback)

other agents have no way to get the object reference

asynchronous message based communication

Running platform

set CLASSPATH environment variable – .jar library launch the main container of the platform with RMA:

java jade.Boot –gui

attach remote containerjava -cp $CLASSPATH jade.Boot -container

-host remoteMachine E.g.:

set CLASSPATH=%CLASSPATH%;.;c:\jade\lib\jade.jar;c:\jade\lib\jadeTools.jar;c:\jade\lib\commons-codec\commonscodec-1.3.jar; c:\jade\lib\iiop.jar

To launch RMA at/from remote machinejava -cp $CLASSPATH jade.Boot –container –host

remoteMachine rma:jade.tools.rma.rma

GeneralitiesGeneralities

6. Agent environments – comparison

Agent Environments (out of 80+)

Aglets (IBM) ?? relatively simple very popular in the past

Voyager (Object Space), Concordia (Mitsubishi) popular in the past

Grasshopper (IKV) worked very well became somewhat popular

NOMADS (UWF) ?? interesting, developed in the academia support?

JADE (Telecom Italia + France Telecom + others) build around FIPA standards long-term € perspective?

Comparing Agent Environments Only an “exercise” to illustrate the

diversity of the environments and their capabilities

Data based on manuals is it reliable? Composed in 2000

only data about JADE is from 2006 Compared

VoyagerConcordiaGrasshopperAgletsJADE

Garbage Collection For some agents “when work done” they

must be removedVoyager deletes an agent as soon as no

references to itGrasshopper agent persists unless it is removed

explicitly by itself, by the external software component, or by the user

Aglets and Concordia agents must be explicitly removed

JADE agent must be explicitly removed or programmed to kill itself

Remote Creation of Agents Capability of creating agents remotely by

sending only necessary parameters

Voyager, Grasshopper, Aglets and JADE support remote agent creation

Concordia does not

Object Mobility Ability to transfer data objects through the

network without the agent traveling itself network traffic reduction

Voyager supports object mobility

others – objects can be “manually” serialized within ACL Messages

Agent Mobility Ability to migrate from one host to another

two types of migration: strong and weak strong migration agent migrates together with its

complete execution state weak migration agent migrates with its data state

(not the stack information)

Voyager, Grasshopper, Concordia, Aglets support weak migration only

JADE not so weak mobility (can program move between behaviors)

NOMADS support strong mobility

Ability to create a copy of itselfresults in “parallel computing”particularly beneficial

data resides in a physically distributed database database resides on a shared memory

multiprocessor

all five environments support cloning

Cloning

Communication Mechanisms Static and mobile agents communicate to solve

problemstypes of communication

synchronous (waits for answer/confirmation) asynchronous (can continue working) dynamic (communication follows the mobile agent)

in Concordia agent groups facilitate synchronous and asynchronous communication

Aglets support synchronous and asynchronous communication

Voyager, Grasshopper and JADE support synchronous, asynchronous, and dynamic communication

Agent teams need support for collaborationConcordia strongest collaboration facilities

agents form agent groups Java distributed objects access them via Java’s Remote Method Invocation (RMI)

facility to perform data exchange

Aglets agents negotiate and schedule distributed meetings

Grasshopper, Voyager and JADE do not support a specific model for collaboration

collaboration via messaging

Agent Collaboration

Security measures are necessary Grasshopper and Voyager support security by

implementing authentication, transmission control and confidentiality of communication

Concordia and Aglets supports authentication and transmission integrity

Concordia provides storage protection for operating without a secure file system

JADE security provided by JADE-S add-on supports: agents authentication, acts permissions, message

integrity and confidentiality; limitations: communication can be only encrypted but not

signed, no mobility related permissions, security features mostly not accessible on MIDP devices

Security

GeneralitiesGeneralities

7. Applications8. Scalability

What does exist in the world? (1) Industry

well kept industrial secretsMitsubishi used Concordia for ???Daimler-Chrysler – production flow control

prototype – 2001 working product – 2005/06 gain 10-20% effectiveness claimed

dynamic transport flow optimization overall cost reduction – about 11%

wineries in East Australia – flow of wine designed using agent tools implemented using C++

What does exist in the world? (2) Academic paper content

lack of regard for FIPA standards (why? better now) frameworks and UML diagrams (tedious, ?easy?) new environments (easy) screen-shots of the interface (easy) initial implementation results (almost easy) no continued implementation effort (difficult) systems with few agents only (why?)

Why Few Agents Only? Size matters!

a lot of agents - how many? a lot of messages - how many?

Need for experiments no “benchmarks” for agent systems agent system “scalability” not “fashionable”

Comments We do not try to establish “benchmarks” Just a “feeling” how large can we get?

Experimental Setup JADE 3.1 agent platform Hardware

Sun workstations UltraSparc III (300 MHz) + 198 Mbytes of RAM

PC’s Pentium II (120 MHz) + 48 Mbytes of RAM

Cisco Switch 100 Mbit/s full backplane

Other machines also tried http://agentlab.swps.edu.pl

Spamming Test (1)

Spamming direction

Computer 1

Spammer

User

Computer 2

Spammer

User

Computer 3

Spammer

User

Each machine spammer and user

Spammers send 5000 ACL messages to all users on all machines

Users process messagesmessages stored in

the queue Message size 300

ASCI characters + ACL wrapper

Spamming Test (2)

821341406248

634847351647

448181288436

313625252175

217501251284

141778244403

87053400342

Receivingtime [ms]

Spamming time [ms]Agent pairs

0

5

10

15

20

25

2 3 4 5 6 7 8

agent "pairs"

tim

e p

er m

essa

ge

[ms]

Spamming

Receiving

Spamming time almost “flat” / receiving practically linear

320,000 messages (100 Mbytes of data) processed (in 14 minutes)

Agent Relay-race (1) Multiple computers Each machine contains relay teams Agents from the Main-Container go to

the next machine and exchange messages to start the next group to move to the third machine…

5 laps Varying number of machines and

number of teams

Agent Relay-race (2)

• Large number of teams / homogeneous / 4 Sun’s

100000

200000

300000

400000

500000

600000

700000

20 40 60 80 100

number of agent-teams

tim

e (m

sec)

Agents in an E-shop• Mother Client sends

Clients (C) to the store• Mother Shop

generates Sellers for each Client

• Seller and Client negotiate and report results of negotiations

• Mother Clients on separate machines

• Results for 30, 40, …, 120 Clients generated

• Scaled to 1450+

0

20000

40000

60000

80000

100000

1 2 3 4 5Number of M otherClient agents

Tim

e (m

s)

Comments On outdated hardware agent systems

“scaled” to: 1400+ agents 320,000 messages

We also did run 2000+ agents on better hardware

All “properties” scaled linearly No restrictions related to JADE but to

computer hardware and network speed

Large agent systems can be implemented!

GeneralitiesGeneralities

9. Coding JADE Agents – preliminary considerations

Agent life-cycle as defined by FIPA

Java JADE: New Classes

jade.core jade.core.Agent jade.core.behaviours

jade.content jade.proto

fipa-request, fipa-query, fipa-contract-net, fipa-subscribe …

…

*.jar libraries:iiop.jarhttp.jar

jadeTools.jarjade.jar--------

commonscodec-1.3.jar

Code – step by step1.Creation of an agent

import jade.core.Agent;public class ExampleAgent extends Agent {  protected void setup() {    // inicialization of an agent  }  protected void takeDown() {     // opt:executes before deleting of agent  }}

An agent terminates when its doDelete() method is calledOn termination the agent’s takeDown() method is invoked (intended to include clean-up operations)

Running In Eclipse (with arguments):-gui <name of instance>:<package>.<Agent localname>

Example:-gui ta:ibspan.lab1.ex2.TruckAgent(true)

___________________________________________________From console (Start, Run..., cmd):

java –classpath <JADE-classes>;. jade.Boot <name of instance>:<package>.<Agent localname>

Code – step by step2.Initialization of an agent

Each Agent instance is identified by an AID (jade.core.AID) An AID composed of a unique name plus additional addresses

<local-name>@<platform-name> An agent can retrieve its AID through the getAID() method of the

Agent class

public class MyAgent extends Agent {protected void setup() {

// TODO Auto-generated method stub//super.setup();System.out.println("My name is " +

getAID().getName());System.out.println("I am glad to see you");

};protected void takeDown() {

// TODO Auto-generated method stubsuper.takeDown();

}}

TA@MIG:1099/JADE

Local names, GUID and addresses The complete name of an agent must be globally

unique The platform name can be set using the –name

option Given the name of an agent its AID can be

created asAID id = new AID(localname,AID.ISLOCALNAME);AID id = new AID(name, AID.ISGUID);

The addresses included in an AID are those of the platform MTPs and are ONLY used in communication between agents living on different FIPA platforms

Code – step by step3. Passing parameters to an Agent

java jade.Boot .... a:myPackage.MyAgent(arg1,arg2)

public class TruckAgent extends Agent {private boolean isFree = false;//@Overrideprotected void setup() {

Object[] args = getArguments();if (args != null && args.length == 1)

isFree = Boolean.parseBoolean((String) args[0]);else {

doDelete();return;

}System.out.println("I am " + (isFree ? "free" : "not free") +" !");}

}

protected void takeDown(){System.out.println("Bye...");}

Ibspan.lab1.ex2

To run w

ithout/w

ith parameters

Add it and check how it

works

Behaviour class The actual job that an agent does is typically carried out

within “behaviors” Behaviours are created by extending the

jade.core.behaviours.Behaviour class To make an agent execute a task it is sufficient to create an

instance of the corresponding Behavior subclass and call the addBehaviour() method of the Agent class

Each Behaviour subclass must implement public void action(): what the behavior actually does public boolean done(): Whether the behavior is

finished An agent can execute several behaviors in parallel,

however, behaviors scheduling is not preemptive, but cooperative and everything occurs within a single Java Thread

Behavior switch occurs only when the action() method of the currently scheduled behavior returns

Code – step by step4. Agent behavior

jade.core.behaviours.Behaviour

Difference between

doDelete(), and

takeDown()

setup()

Agent has been killed (doDelete())

Get the next behavior from the pool of active behaviors

b.action()

b.done()?

Remove currentBehaviorfrom the pool of active behavior

takeDown()

NO

YES

YES

Jade Tutorial: Jade Programming For Beginners, http://jade.tilab.com

Behavior types “One shot” behavior:public abstract class OneShotBehaviourextends SimpleBehaviour {

//Default constructor. It does not set the owner agentpublic OneShotBehaviour() {super();}

//This constructor sets the owner agent for this OneShotBehaviour //code. Parameter a is a “name” of agent this behaviour belongs to

public OneShotBehaviour(Agent a){super(a);

}// The following method makes this behaviour one-shot

public final boolean done(){return true;}

}

Examplepublic class TruckAgent extends Agent {

//Overrideprotected void setup() {Behaviour b = new OneShotBehaviour(this) {

//Overridepublic void action() {

System.out.println("My name is " + getAID().getName());

System.out.println("I am free on Monday!");}

};addBehaviour(b);}

}Ibspan.lab1.ex1

Behavior types (continued) “Cyclic” behaviorspublic abstract class CyclicBehaviourextends SimpleBehaviour {

//Default constructor. It does not set the owner agentpublic CyclicBehaviour() {super();}//This constructor sets the owner agent for this OneShotBehaviour //code. Parameter a is a “name” of agent this behaviour belongs topublic CyclicBehaviour(Agent a){

super(a);}// The following method makes this behaviour cyclic

public final boolean done(){return false;}

}

Example

public class TruckAgent extends Agent {private boolean isFree = false;//Overrideprotected void setup() {

Behaviour b = new CyclicBehaviour(this) {public void action() {

System.out.println("I am " +(isFree ? "free" : "not free") +" !");

}};

addBehaviour(b);}

}

Ibspan.lab1.ex3

Behavior types “Complex” behaviors

FSMBehaviour – models a complex task whose subtasks correspond to activities performed in the states of a Finite States Machine

SequenceBehaviour – models o complex task whose sub-tasks are executed sequentially

ParallelBehaviour – models a complex task whose sub-tasks are executed concurrently

WakerBehaviour OneShotBehaviour The action() and done() methods are already implemented

so that the onWake() method (to be implemented by subclasses) is executed after a given timeout

After that execution the behaviour completes TickerBehaviour CyclicBehaviour

The action() and done() method are already implemented so that the onTick() (to be implemented by subclasses) method is executed periodically with a given period

The behaviour runs forever unless its stop() method is executed

Concurrency in JADE

Agent needs concurrency:to engage in multiple simultaneous conversationsto execute several concurrent tasks

Example - AirConditinioning application, responsible forChecking temperature insideChecking temperature outside

Activities must be performed

concurrently with different freqeuncy

AirConditioning with threads (1)

Let see different approaches to handling concurrencyputting common-used functionality into separate

class: ibspan.lab1.ex3.AirConditioning Threads are managed by JVM in pre-emptive

manner This means threads work in competitive way Look at:

ibspan.lab1.ex4.AirConditioningThreads

AirConditioning with threads (2)

public class AirConditioning {Random gen = new Random(System.currentTimeMillis());float outTemp, inTemp;public AirConditioning(float out, float in) {

outTemp = out;inTemp = in;}

public void checkTempOutside() {float c = (gen.nextFloat() - 0.5f) * 0.1f;outTemp = (1f - c) * outTemp + c;System.out.println("Outside: " + outTemp);}

public void checkTempInside() {float c = (gen.nextFloat() - 0.5f) * 0.05f;inTemp = (1f - c) * inTemp + c; System.out.println(" Inside: " + inTemp);}

}

Outside: 34.614143 Inside: 20.417917Outside: 35.44636 Inside: 20.85639Outside: 34.689354Outside: 34.93689 Inside: 20.849Outside: 33.34642 Inside: 21.218094Outside: 32.28482

AirConditioning with Agents (1)

Each JADE agent is executed in a single Java thread Behaviours are managed by JADE in non-re-emptive

manner This means behaviours work in cooperative way

Every behaviour must release control to allow the other behaviours to be executed

Behaviour switch occurs only when the action() method of the currently scheduled behaviour returns

JADE scheduler carries out a round-robin policy among all behaviours in the ready queue

When the pool of active behaviours is empty the agent enters the IDLE state and its thread goes to sleep

public class AirConditioningAgent1 extends Agent {AirConditioning ac;protected void setup() {ac = new AirConditioning(36f, 20f);

// Construct behaviour checking temperature outsideBehaviour b1 = new SimpleBehaviour(this) {public void action() {

while(true) { ac.checkTempOutside();block(5000);}}

public boolean done() { return false;}};

addBehaviour(b1); // Construct behaviour checking temperature inside

Behaviour b2 = new SimpleBehaviour(this) {public void action() {

while(true) { ac.checkTempInside(); block(8000);}}

public boolean done() {return false;}};addBehaviour(b2);

}

Outside: 35.71913Outside: 35.71527Outside: 35.012707Outside: 36.24267Outside: 36.591988Outside: 37.34552Outside: 37.01548

Infinite loop not allowed in non-

preemptive schedulling

AirConditioning with agents (2)public class AirConditioningAgent2 extends Agent {

AirConditioning ac;protected void setup() {ac = new AirConditioning(36f, 20f);

// Construct behaviour checking temperature outsideBehaviour b1 = new CyclicBehaviour(this) {

public void action() { ac.checkTempOutside();block(5000);}};addBehaviour(b1);

// Construct behaviour checking temperature insideBehaviour b2 = new CyclicBehaviour(this) {public void action() { ac.checkTempInside();block(8000);}};addBehaviour(b2);

}}

Outside: 37.11966 Inside: 19.951431Outside: 36.17769 Inside: 20.149591Outside: 35.71485Outside: 34.88394 Inside: 20.61761

The EndThe End