workshop on high performance, fault- adaptive large scale real-time systems vanderbilt university...
Post on 14-Dec-2015
214 Views
Preview:
TRANSCRIPT
Workshop on High Performance, Fault-Adaptive Large Scale Real-Time Systems
Vanderbilt University
The SRTA Agent Architecture as a Basis for Building Soft Real-Time
Multi-Agent Systems
Victor R. Lesser
Computer Science DepartmentUniversity of Massachusetts, Amherst
November 15, 2002
Acknowledgements
• Byran Horling• Dr. Regis Vincent (SRI)• Dr. Tom Wagner (Honeywell
Research)
• URL:http://mas.cs.umass.edu/~bhorling/papers/02-14.ps.gz
Outline
• Background/Motivation• EW Challenge Problem • Approach to Soft Real-Time• Approach to Building MAS
• SRTA Agent Architecture
• Experimental Evaluation
• Summary
Long Term Motivation
• Development of domain-independent techniques (toolkits) for coordinating the soft real-time activities of teams of cooperative agents?
• Ease the construction of complex, multi-agent applications that operate in a coherent manner
• Avoid reproducing for each application the complex reasoning involved in soft real-time control and in coordinating the activities of agents
DARPA EW Challenge Problem: Distributed Sensor Network
•Small 2D Doppler radar units
–Scan one of three 120 sectors at a time
Commodity Processor associated with each radar
•Communicate short messages using radio
•Triangulate radars to do tracking
Approach to Soft Real-Time: Design-to-Time
• “Given a time bound, dynamically construct and execute a problem-solving procedure which will (probably) produce a reasonable answer with (approximately) the time available.” (D’Ambrosio)
• Involves elements of planning (deciding what to do) and scheduling (deciding when to perform particular actions).
TÆMS: A Domain Independent Framework for Modeling Agent Activities
• The top-level goals/objectives/abstract-tasks that an agent intends to achieve
• One or more of the possible ways that they could be achieved• abstraction hierarchy
(HTN) whose leaves are basic action instantiations, called methods
• A precise, quantitative definition of the performance (Qaf’s) • solution quality, cost, time
and resource usage.
Recommend a High-End PC System
Make Decision
MoneyResource
Build ProductObjectsOutcomesNum Prod 1-4Num Prod 5-8Num Prod 9-12Num Prod ...
Get Basic ProductInformationQuery & ExtractVender mQuery & ExtractPossible Maker n
Gather ReviewsSearch & ProcessZDnet Reviews Search & ProcessPC World
Query & ProcessConsumers Reports
q_sum_all()q_sum() q_sum()
q_seq_last()
q (10% 0)(90% 10)c (100% 0)d (10% 2min)(10% 2.5min)(80% 3min)q (20% 0)(80% 8)c (100% 0)d (50% 1min)(50% 2min)Query & ExtractPC Connection Query& ExtractNECX q (25% 0)(75% 20)c (100% $2)d (90% 3)(10% 5)
q (15% 0)(75% 10)c (100% 0)d (30% 3min) (30% 4min) (40% 5min)q(..), c(..), d(..)q(..), d(..), c(..)
consumes $2limitsq multiplier (100% 0)c multiplier (x)d multiplier (x)
q(..), c(..), d(..)
facilitates & hindersfacilitates & hindersq multiplier (100% +20%)d multiplier (100% +20%)
TaskMethodResource nleTask nleSubtask RelationKey
enables.........q_sum()
Soft Real Time Control -- Different Paths for Achieving Task -- “BUILD PRODUCT
OBJECTS”
• Schedule A - Client has no resource limitations; maximize quality• Query-and-Extract-PC-Connection, Query-and-Extract-PC-Mall, Search-and-Process-
ZDnet, Query-and-Process-Consumers-Report (Expected Q=55.3,C=2, D=11.5)
• Schedule B - Client is interested in a free solution• Query-and-Extract-PC-Connection, Query-and-Extract-PC-Mall, Search-and-Process-
Zdnet (Expected Q=33.2,C=0, D=8.4)
• Schedule C - Client request an even trade-off between quality, cost and duration
• Query-and-Extract-PC-Connection, Search-and-Process-Zdnet (Expected Q=22.4,C=0, D=5.6)
• Schedule D - Client wishes to maximize quality while meeting a hard deadline of 7 minutes• Query-and-Extract-PC-Mall, Query-and-Process-Consumers-Report
(Expected Q=25.9,C=2, D=6)
Examples of Schedules Produced by the Design-To-Criteria (DTC) Scheduler
Representing Coordination Patterns Among Agents
MoneyResource
enables
enablesRecommend a High-End PC System
Make Decisionq_sum_all()q_seq_last()Build ProductObjects
Get Basic ProductInformationNon-local Task Gather ReviewsNon-local Task
Get Basic ProductInformationQuery & ExtractVender mQuery & ExtractPossible Maker n
q_sum()Query & ExtractPC Connection BuyInformation
Gather ReviewsSearch & ProcessPC Worldq_sum()Search & ProcessZDnet ReviewsQuery & ProcessConsumers Reports
task contractingtask contracting
consumes / limitsconsumes/ limits
Task Agent
InformationAgentInformation Agent
Approach to Decomposing A Problem into Agents
• Sophisticated/Highly Competent Agents• Concurrent goals, goals are time and resource sensitive,
goals have varying utilities
• Goals have alternative ways of being solved that produce differing levels of utility and consume differing amounts of resources
• Not all goals necessarily need to be solved
• (Sub)Goals spread across agents are interdependent• Contention for scarce resources
• Contributing towards the solution of a higher-level goal
• Hard and soft constraints
• Sufficient computational/communication resources to do “some” reasoning about coordination• Medium granularity domain tasks
What about simpler agents?
• Activities of simple,single-threaded agents become the goals of sophisticated agents with dedicated processing resources
• Sophisticated agents do the selection, multiplexing, scheduling, coordination and distribution of goals• Contrast with O.S. doing the scheduling without context
Sector Manager
Tracking Manager
Tracking Agent
Scanning Agent
Approach to Soft, Real-Time Distributed Coordination/Resource Allocation
• Structured as a distributed optimization problem with a range of “satisficing” solutions
• Adaptable to available time and communication bandwidth
• Responsive to dynamics of environment• Organizationally constrained — range of agents
and issues are limited• Can be done at different levels of abstraction• Does not require all issues to be resolved to be
successful — resource manager agents able to resolve some issues locally
Layered Agent Architecture
• Domain analysis and goal formulation• Organization-level resource allocation
• Agent-level resource allocation• Constraint discovery and satisfaction
• Intra-agent organization and communication
• Environmental access points
Problem Solver/ Negotiation
Soft Real Time Architecture
Java Agent Framework
JAF: Java Agent Architecture• Component-based agent design• Attempt to maximize code reuse.
Execute Control CommunicatePulse Actions Scan Scheduler
Problem Solver
Execute Control Communicate
Resource Modeler Directory ServiceLog SchedulerStateSensor Observe
• Interfaces are hidden by JAF.Interfaces are hidden by JAF.•Radsim/ RF communication/ sensorRadsim/ RF communication/ sensor
SRTA: Soft Real-Time Agent Architecture
• Facilitates creation of multi-resource management agents
• Basis of building complex “virtual” agent organizations
• Allows for abstract negotiation — maps abstract assignment into detailed resource allocations
• Ability to resolve conflicts locally that are not resolved through negotiation
These are key to building soft real-time distributed allocation policies
Soft Real-Time Control Architecture
Resource Modeler
Conflict Resolution Module
Task Merging
Problem solverPeriodic Task Controller
TÆMS Library
Cache Check
DTC-Planner
Partial Order Scheduler
Parallel Execution Module
LearningUpdateCache
CacheHit
Linear Plan
TAEMS-Plan Network/Objective
Goal Description/Objective
Parallel ScheduleScheduleFailure
Results
Update Expectations
Schedule Failure
OtherAgents
Schedule
Resource Uses
MultipleStructures
Negotiation(e.g. SPAM)
Commitments/Decommitments
Schedule failure/Abstract view
• Operates at 50 to 100ms cycle time• Written in JAVA except for Planner in C++
• Uses domain-independent, quantitative representation of agent activities -- TÆMS• Scheduling of multiple activities that have deadlines
and are resource sensitive• Can choose among alternative ways of achieving
activities that trade off decreased utility for lower resource consumption
• Responds to uncertain conditions without the need for complete re-planning/scheduling of activities
Characteristics of Soft Real-Time Control Architecture
Addressing Real Time – Direct
• Direct technologies - making it possible• DTC (Design-To-Criteria) planner
• TÆMS HTN for representing alternative plan options using quantitative information
• Create appropriate plan given time, resource costs and quality constraints
• Partial order scheduling creates “loose” schedules which can be quickly shifted to real-time constraints.• Avoids constant re-planning
• Allows parallel execution and resource usage.
• Modeling of some meta-level activities (e.g. negotiation) permit more direct reasoning of time allocation.• We do not model scheduling costs yet.
• Learning component discerns actual execution characteristics so future actions can be better modeled.
Addressing Real Time – Indirect
• Indirect technologies - making it easier• Periodic commitments reduce the need
for re-negotiation.
• Scheduled caching reduces the need to call DTC
• Piecemeal addition and removal of tasks eliminates the need for constant dramatic rescheduling and re-planning.
Scheduling
• Partial-order Scheduler uses a “sliding” mechanism, coupled with a resource modeler, to quickly shift scheduled tasks.
• Action start time uncertainty - real time.
• Duration uncertainty.
• Commitments have a “window” of time in which the agent can perform them.
• Precise action scheduling is left to the discretion of the performing agent.
Set-parameters
First task to achieve
enables2 enables3
lock
Track LowTrack MediumTrack High
Send Results
Sensor
Task 1Q_min
TaskQ_max
Deadline: 3000
RF
2 Other tasks to achieve
Enables1
Init Calibrate Send-Message 1
lock1
Q_min
Enables4Task2Q_min
Task3
Negotiate-Tracking Send-tracking-Info
Sensor
Enables4
RF
Reacting to Unexpected Changes
500 1000 1500 2000 2500 3000 3500 4000
Init
Calibrate
Send Msg
Set-Parameters Track-Medium Send-Results
Send-Info-TrackingNegotiate-Tracking
500 1000 1500 2000 2500 3000 3500 4000
Init
Calibrate
Send Msg
Set-Parameters Track-Medium Send-Results
Send-Info-TrackingNegotiate-Tracking
500 1000 1500 2000 2500 3000 3500 4000
Init
Calibrate
Send Msg
Set-Parameters
Negotiate-Tracking
time
time
timeSend-Info-Tracking
Send-Results
•Analogous reactions also take place within the periodic task controller• Slot-based scheduler used to facilitate repetitive actions
Track-Medium
Meta-Level Costing
• Typical scheduling reasons about primitive actions.• This only accounts for some percentage of the
agent’s time.• So called meta-level activities (e.g. negotiation,
scheduling, planning) use significant resources but are usually not accounted for directly.
• Without accountability, these activities can interfere with the actions and commitments currently scheduled over.
Meta-Level Costing (cont’d)
• To completely reason about all the agent’s actions, we must: • Directly and Indirectly incorporate the activities in
plans. • Derive expected costs for these activities.• Use this information when generating schedules.
• We are currently using a representation of negotiation activities in some our task structures• Future goal is to more directly account for
activities like planning and scheduling.
Reducing Scheduling Overhead
• Activity parallelism learning• Anticipation of converting linear plan to
parallel plan if resources are available
• Schedule caching
• Adjustable time granularity• Responsiveness vs. meta-level overhead
Activity Parallelism Learning
• The plan is first scheduled…
• The schedule is analyzed for parallel actions
• These are used to form sets of “hints,” associated with the current resource context
• A hint is actually just a mutual facilitates relationship
• These hints are later applied to the task structure before planning
• The facilitates tells DTC that if A is run first, B will have a duration of zero (or vice versa)
• This technique required no changes to DTC
• This can result in better plan selection.
q_min
Plan B
A B C1 C2
q_max
A
B
C1
E
Deadline
A B C1
Deadline
DTC Schedule
Actual Schedule
q_min
Plan B
A B C1 C2
q_maxE
A
B
Deadline
A B
Deadline
DTC Schedule
Actual ScheduleC2
C2
F F
Original Plan
Plan With Hints
Schedule Caching
• Agents in repetitive environments must frequently address goals which have previously been seen.• Ex: In sensor environment, “Scan-Sector” or “Perform-Track-
Measurement”• Results from prior planning can be reused
• A key is generated for each task structure• Incorporates method names and expectations, interrelationships,
normalized deadlines, etc.• Works correctly with activity parallelism hints
• If results exist from a task structure with the same key, that plan is used instead of calling DTC
• DTC is an external C++ binary, requiring file reads and writes, so savings are significant
Adjustable Time Granularity
• The millisecond time line is divided into coarser increments.• Does not significantly degrade the agent’s ability to meet “wall
clock” time deadlines• (assuming success within a reasonable grain size)
• It does decrease the number of rescheduling events which are needed
• Since the agent was already operating effectively at a coarse timeline, this technique has no new drawbacks.
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 1 2 3 4 5
Real Time
Perceived Time
Real Time
Perceived Time
Deadlines
Actions
Deadlines
Actions
Effects of Schedule Caching
• Periodic tasks and methods with deadlines are more achievable
• As a result, track updates increase
Effects of Changing Granularity
Rescheduling attempts decrease, reducing overheadMethods with deadlines and periodic tasks are satisfied more often30-40 seems ideal; above that, the coarse granularity decreases agent
responsiveness
Summary
• SRTA architecture is a powerful tool for building soft real time agent organizations
• Sophisticated soft real time agent control is practical by exploiting a variety of mechanisms• to speed up the planning, scheduling and
rescheduling cycles
top related