dr. muhammad adnan hashmi

An Agent Oriented Programming Language integrating Temporal

Planning and the Plan Coordination Mechanisms

Dr. Muhammad Adnan Hashmi

2

Outline Introduction

Background Problem Overview

Plan Coordination Mechanisms Coordinated Planning Problem Proactive-Reactive Coordination Problem

P-CLAIM: AOP Language supporting Temporal Planning Language Definition Planning Mechanism

Conclusion

3





Conclusion

4

What is an Agent? An agent is a computer system capable of autonomous

action in some environment in order to meet its design objectives (goals).

Achievement of goals is important for agents

5

Planning Finding a sequence of actions that takes agent from

Initial State to Goal State

Initial state:clear(c),on(c,a),ontable(a),clear(b),ontable(b),handempty

ca b

Goal:on(a,b),on(b,c) c

ab

ca b ca b ca

bc

ab

6

Multi-Agent Planning Multiple agents sharing the same environment

Independent planning is not sufficient Coordination of plans is vital

Coordination of plans Removing conflicts (negative interactions) Utilizing help relations (positive interactions)

A

B

DA

DB

A

B

DA

DB

7

How to Program Agents? Agent Oriented Programming (AOP) Languages allow to

program intelligent and autonomous agents

Main Characteristics Mental State: Beliefs, Goals, Commitments Reasoning Mechanism Capabilities, Services Communication

Some Languages Agent-0 [Shoham 1993], 2APL [Dastani 2008], AgentSpeak (L)

[Rao 1996]

8





Conclusion

9

Problem Statement Current AOP languages

Follow a reactive (PRS based) approach Do not support temporal planning

Only a few support planning

Problems Execution without planning may result in the goal failures

Agent can reach a dead end Conflicts can arise among different agents Actions’ duration is important

Real world actions take place over a timespan

10

Objectives Propose a programming language that endows agents with

planning skills Has temporal planning Deals with uncertainty of the environment Incorporate reactivity by dealing with on the fly goals having

different priorities

Propose coordination mechanisms for the plans having different priorities

11





Conclusion

12

Assumptions Two agents α and β sharing the same environment

Agent α having higher priority (reactive) goals Agent β having normal priority (proactive) goals

Actions have: Preconditions Add effects Delete effects

Two possible conflicts among plans Causal link threat Parallel actions interference

13

Two Possible Conflicts Causal Link (A1, A2, p)

Action A1 adds an effect p Action A2 needs this effect No action between A1 and A2 adding p

Causal Link Threat If an action A deletes p and lies between A1 and A2, then A

threatens the causal link (A1, A2, p)

A1 A2p

A ¬p

Threat

14

Two Possible Conflicts Causal Link (A1, A2, p)

Action A1 adds an effect p Action A2 needs this effect No action between A1 and A2 adding p

Parallel Actions Interference Actions A1 and A2 lie in parallel Either one of them deletes the preconditions or add effects of the

other

A1

A2

¬p

p

A1

A2

p

¬p

15





Conclusion

16

Coordinated Planning Problem Prerequisite:

Plan Pα of Agent α

Our Aim: Compute a Plan Pβ for Agent β

Has no conflict with Pα

Avails the cooperative opportunities offered by Pα

Solution: Non Temporal Domains µ-SATPLAN Temporal Domains Coordinated-Sapa

SATPLAN [Kautz and Selman

2006]A Classical Planner that Finds Optimal

Plans in Non-Temporal Domains

18

SATPLAN

Compiler(encoding)

satisfyingmodel

Plan

Increment plan lengthIf unsatisfiable

Planning Problem • Init State• Goal• Actions

CNF Simplifier(polynomial inference)

Solver(SAT engine/s)

Decoder

CNF

Propositional formula in conjunctive normal form (CNF)

We Use GraphPlan Encoding

19

Constructing the planning graph Level P1: All literals from the initial state Add an action in level Ai if all its preconditions are present in

level Pi Add a proposition in level Pi if it is the effect of some action in

level Ai-1 Maintain a set of exclusion relations to eliminate incompatible

propositions and actions

20

GraphPlan Encoding

Fact Act1 Act2

Act1 Pre1 Pre2

¬Act1 ¬Act2

Act1

Act2

Fact

Pre1

Pre2

Can create such constraints for every node in the planning graph

Only involves facts and actions in the graph

µ-SATPLAN

An Extension of SATPLAN that Computes Coordinated Plans in Non-

Temporal Domains

22

Handling Causal Link Threat

While constructing the planning graph for Agent β, don’t add an action O at level Ai if It has an effect ¬p, and There is a causal link (Aj, Ak, p) in plan Pα, and j ≤ i ≤ k

Action O threatens Causal Link (Aj, Ak, p)

23

Handling Positive Interactions & Parallel Actions Interference For each time step i in the plan of Agent α, create an

action NoName(i)

Add(NoName(i)) All the effects added by Pα at time step i

Del(NoName(i)) All the effects deleted by Pα at time step i

Pre(NoName(i)) All the preconditions of actions of Pα at time step i

Explicitly add all the NoName actions in the planning graph of Agent β

24

Handling Positive Interactions Pα = {α1(0), α2(0), α3(1)} Eff(α1) = a0, Eff(α2) = a1, Eff(α3) = a2

Eff( NoName(0) ) = {a0,a1}, Eff( NoName(1) ) = {a2}

a0

a1

a2

a4

a5

a6a7

a8

Level 0 Level 1 Level 2

a4

a6

NoName(0)

β1

a6

a4

a0

a1

NoName(1) a2

a0

a1

β2

a5β3

a6

a5

a4

a7

β4

Planning Graph of Agent β

25

Handling Positive Interactions Partial CNF Sentence

a8 β4 β4 a0 a7 a0 β2 NoName(0) a7 β3 β2 a5 …

Add NoName actions as unary clauses to CNF NoName(0) NoName(1)

Problem Solution

a0

a1

a2

a4

a5

a6a7

a8

Level 0 Level 1 Level 2

a4

a6

NoName(0)

β1

a6

a4

a0

a1

NoName(1) a2

a0

a1

β2

a5β3

a6

a5

a4

a7

β4

26

Example Plan Generated (Logistics)

Time Action Performed

0 DRIVE-TRUCK (TRUCK1, LOC1-1, LOC1-2, CITY1)

1 LOAD-TRUCK (PACKAGE1, TRUCK1, LOC1-2)


1 LOAD-PLANE (PACKAGE2, PLANE1, LOC1-1)


2 FLY-PLANE (PLANE1, LOC1-1, LOC2-1)

3 UNLOAD-PLANE (PACKAGE2, PLANE1, LOC2-1)

3 UNLOAD-TRUCK (PACKAGE1, TRUCK1, LOC1-1)







Time Action Performed


1

2











Pα Pβ

27





Conclusion

28

Proactive-Reactive Coordination Prerequisite:

Reactive plan Pα of Agent α Proactive plan Pβ of Agent β

Our Aim: Modify plan Pβ such that:

It has no conflict with Pα

Avails the cooperative opportunities offered by Pα

Solution: Plan Merging Algorithm

29

Case Study Tasks of Rescue Agent

Rescue the Victims Tasks of Analyzer Agent

Analyze the goal cells Call the central agent

Constraints One agent in a cell Hyper energy cells

Needs fuel or energy to enter Agent should have key to open door

Rescue Agent : αAnalyzer

Agent : β

D

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15

CBA

30

Conflict Resolution Threat-Repair Link (A1, A2, p)

Action A1 deletes p A2 is a subsequent action and adds p

A1 is called Threat Action A2 is called Repair Action

B1 B2p

A1 -p A2 p

Threat

Repair

31

Valid and Possibly Valid Time Stamps Possibly Valid Time Slot for an action A

All preconditions are met No parallel actions interference

P[1] a bh P[2] b c

-d P[3] c e P[4] e f P[5] f g

P[1] b d-h

P[6] g i P[7] i h g

Valid Time Slot for an action A All preconditions are met No parallel actions interference Either:

No causal link threat Repair Action exist before the deadline

P[1] a bh P[2] b c

-d P[3] c e P[4] e f P[5] f g

P[1] b d-h

P[6] g i P[7] i h g

P[2] d k P[3] k h

32

Plan Merging Algorithm

Fix all the actions of Reactive Plan Pα on timeline

For every action CA of Proactive Plan

Search for the first Possibly Valid Time Slot T on timeline

Reason about the time slot T There could be 5 cases at T

33

Plan Merging AlgorithmCase 1: No causal link threat by CA at T

Assign Time Slot T to CA

EXAMPLE Current Action: Move(A1, A2)

Returned Time Slot: 0 - 5 Any Threat? : No Assign Time Slot 0 – 5 to CA

D

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15

CBA

34

Plan Merging AlgorithmCase 2: CA threatens a Causal Link but Repair Action exist

Assign Time Slot T to CA Save a Possible Threat <ThreatAction, RepairAction, Deadline>

EXAMPLE Current Action: Move(A4, A5)

Time Slot: 20 - 25 Any Threat? : Yes (Agent α needs A5 at time 40-45) Repair Action: Move(A5, A6) Assign Time Slot 20 - 25 to Move (A4, A5) Save <Move(A4, A5), Move(A5, A6), 40>

D

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15

CBA

35

Plan Merging AlgorithmCase 3: It is a Repair Action but can not meet a deadline of

some Threat Action Backtrack to the Threat Action,find another time stamp

EXAMPLE Current Action: Move (A8, A9)

Returned Time Slot: 50 - 55 Any Threat?: Yes (Agent α needs A9 at 85-110) Repair Action : Move (A9, B9) Save <Move(A8, A9), Move(A9, B9), 85>

Next Action: AnalyzeCell (A9) Time Slot Assigned: 55 - 70

D

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15

CBA

36

Plan Merging Algorithm Next Action: CallCentral (A9)

Time Slot Assigned: 80 – 90 Next Action: Move (A9, B9)

Is it a Repair Action? : Yes Meet all deadlines?: No (Agent α needs A9 at 85) Backtrack to action Move(A8, A9)

Find another Time Slot New Time Slot: 110 – 115 (Valid Time Slot)

D

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15

CBA

Attention

37

Plan Merging AlgorithmCase 4: All the effects of CA are already achieved WHAT TO DO? Mark CA as redundant POST PROCESSING Remove all redundant actions from the plan Recursively remove all actions which achieve only the

preconditions of removed action

38

Plan Merging AlgorithmEXAMPLE Current Action: OpenDoor (C11)

Returned Time Slot: 172 - 175 Redundant(OpenDoor(C11)) true

Because openedDoor(C11) is true at time 172

When the plan is returned Remove OpenDoor(C11) from plan Also remove TakeKey(C11, key1) from plan

D

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15

CBA

39

Plan Merging AlgorithmCase 5: Action CA’s preconditions can not be achieved

Remove action CA from the plan and compute a plan to achieve effects of CA

I = State just before CA G = Effects (CA)

Plan should have no conflict with Reactive Plan Pβ and if CA is a repair action, repair effects must meet their deadline

ReplacementPlan = Coordinated-Sapa (I, G, Pβ)

If a plan is returned, replace the removed actions with the plan

If a deadline is violated, backtrack to the threat action

If no plan possible, then remove another action CA + 1

G = G U Effects (CA + 1) \ Pre (CA + 1)

Use Coordinated-Sapa

40

Plan Merging AlgorithmEXAMPLE Current Action: TakeEnergy(B13, energy1) Preconditions can not be achieved

Repair the plan

D

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15

CBA

41

Plan Repair Algorithm Create a CPP by removing TakeEnergy(B13, energy1)

I = { at(β, B13), at(energy1, B13), at(energy2, B13) } G = { hasEnergy(β, energy), at(β, B13)}

Call Coordinated-Sapa to solve this CPP Coordinated-Sapa returns fail Why? energy2 is also needed by Agent α

D

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15

CBA

42

Plan Repair Algorithm Create another CPP by removing Move(B13, A12)

I = { at(β, B13), at(energy1, B13), at(energy2, C15) } G = { at(β, A12) }

Call Coordinated-Sapa to solve this CPP A plan is returned to enter A12 by taking the fuel from D14

POST PROCESSING This plan will become a replacement for both

TakeEnergy(B13, energy1) and Move(B13, A12)

D

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15

CBA

43





Conclusion

44

An AOP language having: Cognitive aspects specific to intelligent agents Communication primitives Mobility primitives Temporal planning capability

P-CLAIM Agent: Is autonomous, intelligent and mobile Has a mental state containing knowledge, goals, and capabilities Is able to communicate with other agents Entails a planning based behaviour Achieves goals based on their priorities Maintains the stability of the plan in the dynamic environments

P-CLAIM

45

Defining P-CLAIM Agent

defineAgent agentName{ parent = null | agentName ; knowledge = null ;| {knowledge1; …; knowledgem} goals = null ;| {goal1; … ; goaln} activities = null ; {activity1 … activityo} actions = {action1 … actionp} agents = null ; | {agName1, agName2, …, agNameq}

}

46

Activities


}activityity = name { message = null | message ; conditions = null | condition ; do { process } effects = null ;| { effect1 ; …; effectf }

}

47

Actions


}action = name { message = null | message ; conditions = null | condition ; do { process } duration = dur ;

}

48





Conclusion

49

Agent Definition to Planning (Translator)

Translator(JavaCC)

Knowledge Goals Activities Actions

Initial State Goals Methods Operators

Problem File Domain File

Agent Description File

Planner

50

Agent Life Cycle

51

Messages Handler

52

Waits for messages from other agents

Message is a request to achieve a goal? Assigns priority to the goal Reactive Goal puts it in Global Reactive Goals (GRG) queue Proactive Goal puts it in Global Proactive Goals (GPG) priority

queue

Message is an information? Store the information in the knowledge base of the agent

Messages Handler

53

Planner

54

Temporal Converter

Compute Plan

Main Algorithm

GRG GPG

1- Fetch goals one by one

from GRG and GPG and calls Compute_Plan to compute a plan for the

goal

2- GPG Goals are accessed

only when GRG is empty

3- Sends a suspension

signal to Executor if the

goal is reactive i.e. from GRG

Planner

55

Plan Computation JSHOP2 algorithm [Nau et el. 2003]

is used to compute a totally ordered plan for each goal

An HTN planning algorithm

Decomposes the task into sub-tasks by applying methods

Recursively applies the same procedure on every composite sub-task until there are only primitive tasks

T1

M1

T11 T12

M12M11

56

Temporal Converter

Input to procedure A totally ordered plan Actions’ information (Add, Del, Pre, Durations)

Output of procedure A position constrained parallel plan

Every action is assigned a time stamp Multiple actions can possibly lie in parallel

57

P[1]a-a+d+e+f

P[2]f-f+g+h

P[3]e +i

P[4]h -h+j

P[5]gi +k

P[6]k +l P[7]l +m P[8]m +n+c P[9]

c mn

-c

P[10]e-k-c+h

P[1]a-a+d+e+f

P[2]f-f+g+h

P[3]e +i P[4]h -h+j P[5]g

i +k P[6]k +l P[7]l +m P[8]m +c P[9]c m -c P[10]e

-k-c+h

Input Plan

Output Plan

Makespan Gain: 30%

Temporal Converter (Example)

Makespan =150

Makespan =105

58

Merging the New Plan to Global Plan

59

Merging the New Plan to Global Plan

Planner

Append at the end of PexecMerge at the start of Pexec

Proactive GoalReactive Goal

Plan Under Execution (Pexec)

60

Schedule Handler and Executor

61

Schedule Handler and Executor

62

Plan Mender

63

Plan Mender

SP(Pexec[1]) SP(Pexec[2])SP(Pexec[3])

GSP(Pexec[4])

SW

Compute Plan Using Sapa

No Plan

Compute Plan Using Sapa

ReplacementPlan ContinuationPlan

Pexecnew = ReplacementPlan + ContinuationPlan

64





Conclusion

65

Conclusion An agent oriented programming language supporting:

Temporal Planning Plan Repairing Dealing with different priority goals

Coordinated Planning Problem Computing plan while coordinating it with another plan

SATPLAN µ-SATPLAN Sapa Coordinated-Sapa

Proactive-Reactive Coordination Problem Modifying a plan to remove conflicts with a higher priority

plan Plan Merging Algorithm

66

Publications1. A. El Fallah Seghrouchni, M. A. Hashmi, “Multi-Agent Planning”, Book

chapter in Software Agents, Agent Systems and Their Applications. M. Essaaidi et al. (Eds.), 2012, IOS Press.

2. Y. Dimopoulos, M. A. Hashmi, P. Moraitis, “µ-SATPLAN: Multi-Agent Planning as Satisfiability”, Knowledge Based Systems Journal, 2011

3. M. A. Hashmi, A. El Fallah Seghrouchni, “Merging of Temporal Plans supported by Plan Repairing”, Proceedings of 22nd IEEE International Conference on Tools with Artificial Intelligence (ICTAI 2010), Aras, France

4. M. A. Hashmi, A. El Fallah Seghrouchni, “Coordination of Temporal Plans for the Reactive and Proactive Goals”, Proceedings of IEEE/WIC/ACM International Conference on Intelligent Agent Technology (IAT’ 10), Toronto, Canada

5. Y. Dimopoulos, M. A. Hashmi, P. Moraitis, “Extending SATPLAN to Multiple Agents”, Proceedings of 30th SGAI International Conference on Artificial Intelligence 2010, Cambridge, UK.

6. M. A. Hashmi, A. El Fallah Seghrouchni, “Temporal Planning in Dynamic Environments for P-CLAIM Agents”, In proceedings of Languages, Methodologies and Development Tools for Multi-Agent Systems (LADS’09), Torino, Italy, Springer-Verlag.

67

Publications7. M. A. Hashmi, A. El Fallah Seghrouchni, “Temporal Planning in Dynamic

Environments for Mobile Agents”, Proceedings of International Conference on Frontiers of Information Technology (FIT’ 09), Abottabad, Pakistan, Publisher ACM press.

8. M. A. Hashmi, “A Planning Component for CLAIM Agents”, In the proceedings of 17th International Conference on Control Systems and Computer Science, Bucharest, Romania, Volume 2, pages 485-492, Politehnica Press

68

Thanks for your attention!!!

dr. muhammad adnan hashmi

Documents

plan coordination mechanismsdr

p action a1

temporal planningonly

program agents

effect p action a2

reactive prs

p causal li

fly goals