1 05/01/02 ISORC 2002

BBN TechnologiesJoe Loyall

Rick Schantz, Michael Atighetchi, Partha Pal

Packaging Quality of Service Control Behaviors for Reuse

ISORC 2002

April 29-May 1, 2002

2 05/01/02 ISORC 2002

Trends Motivating Development of QoS-Enabled, Adaptive Middleware

• Trends in technology development– Hardware keeps getting smaller, faster, and cheaper

– Software keeps getting larger, slower, and more expensive

• Challenges in Software Development– Building distributed systems is difficult

– Simultaneously the two dimensions of» Wider distribution across WANs, including wireless and satellite links

» Smaller embedded environments with strict QoS and resource constraints

have made distributed systems development increasingly difficult

C IN CJ 2

C IN C O p e r a t io n sP la n n in g G r o u p

O P S / IN T E LW O R K S T A T IO N

T A R G E T

C J T FP la n n e r s

T A R G E T

M is s io n sC e n te r s o f G r a v i t yT a s k r e f in e m e n tF o r c e s r e f in e m e n tP h a s e s r e f in e m e n tC O A e v a lu a t io nC O A s e le c t io n

F L T C IN C M a r i t im eP la n n in g C e n te r

T M S C A S E S & H P C

M a r i t im e & A irC a m p a ig n A s s e s sm e n t

J O IN T F O R C E M A R IT IM EC O M P O N E N T C M D R

C A S E S T A R G E T

T a s k r e f in e m e n tIn t e g r a t e d T a r g e t P r io r i t ie sF o r c e s r e f in e m e n tS c h e d u le r e f in e m e n tC O A e v a lu a t io n

J O IN T F O R C E A IRC O M P O N E N T C M D R

T A R G E T A C P T

ID B

M is s io n sC O A sS t r a te g y

C r is isa s s e s s m e n t

C O A e v a lC O A e v a l A irM a r

X ID B

T a r g e t N o m in a t io n L is tM a s te r A t ta c k P la nA t ta c k P la n S ta tu s

J F A C CC o m b a t O p s

T a r g e t N o m in a t io n L is tW e a p o n e e r in g

X ID B

A P S /F L E X

T A M P S /C O M P A S S

J M C ISID B

A ir T a s k in gO r d e r ( A T O )

N A T OO P S / IN T E LW O R K S T A T IO N

R e a r - e c h e lo nS im C e n t e r

t oS H A P E

M a s te rT a r g e tL is t

A P S /F L E X R A A P

R e f in e m e n t& e v a l

c o l la b o r a t iv e d e v e lo p m e n t

r e h e a r s e

R e f in e m e n t& e v a l

A T O

T a r g e t d a t a / W e a p o n e e r in g

4 05/01/02 ISORC 2002

The Quality Objects (QuO) Framework Supports Development of Distributed Applications with QoS

The QuO framework provides• Separation of concerns between software functional properties and QoS needs

• Consistent interfaces for QoS measurement and resource management control

• Standard middleware interfaces between application and QoS-provider layers

• Facilities to enable application- and system-level adaptation

QuO is being developed as a common approach to adaptable QoS for a number of projects focusing on different QoS dimensions:

• Dependability, replication, group communication (U.Illinois, Cornell, UCSB)

• Managed bandwidth, resource reservation (CMU, Columbia Univ., OOMWorks)

• Real-time behavior (Washington Univ in St. Louis, OOMWorks)

• Security, access control, survivability, intrusion detection (TIS)

• Integrating different dimensions in an application, e.g., real-time and dependability

• Suitability for supporting different approaches to a dimension

5 05/01/02 ISORC 2002

QuO Adds Specification, Measurement, and Adaptation into the Distributed Object Model

ApplicationDeveloper

MechanismDeveloper

CLIENT

Network

operation()

in args

out args + return value

IDLSTUBS

IDLSKELETON

OBJECTADAPTER

ORB IIOP ORBIIOP

CLIENT OBJECT(SERVANT)OBJECT(SERVANT)

OBJREF

CLIENT

DelegateContract

SysCond

Contract

Network

MECHANISM/PROPERTYMANAGER

operation()

in args

out args + return value

IDLSTUBS

Delegate

SysCond

SysCond

SysCond

IDLSKELETON

OBJECTADAPTER

ORB IIOP ORBIIOP

CLIENT OBJECT(SERVANT)OBJECT(SERVANT)

OBJREF

ApplicationDeveloper

QuODeveloper

MechanismDeveloper

CO

RB

A D

OC

MO

DE

LQ

UO

/CO

RB

A D

OC

MO

DE

L

6 05/01/02 ISORC 2002

CLIENT

DelegateContract

SysCond

Contract

Network

MECHANISM/PROPERTYMANAGER

operation()

in args

out args + return value

IDLSTUBS

Delegate

SysCond

SysCond

SysCond

IDLSKELETON OBJECT

ADAPTER

ORB IIOP ORBIIOP

CLIENT OBJECT(SERVANT)OBJECT(SERVANT)

OBJREF

QuO Provides In-Band and Out-of-Band Measurement

• In-band measurement handled by instrumentation– A structure is transparently

passed along with the method call/return

– Information can be inserted, read, and processed to record and evaluate method call statistics (e.g., the time spent in marshalling)

• Out-of-band measurement provided by system condition objects

Mechanism Developer

Specialized ORBs or Services

SimpleValue

MeasuredValue

(Sensor)

ComposedValue

ApplicationDeveloper

QoSDeveloper

QuO Kernel

RSVPController

ControlValue

StatusValue

CORBA ObjectDeviceStatus

Service

ControlValue

7 05/01/02 ISORC 2002

QuO Provides In-Band and Out-of-Band Adaptation and Control

• In-band adaptation provided by the delegate and gateway

– A delegate decides what to do with a method call or return based upon the state of its contract

– Gateway enables control and adaptation at the transport layer

• Out-of-band adaptation triggered by transitions in contract regions

– Caused by changes in the system observed by system condition objects

CLIENT

DelegateContract

SysCond

Contract

Network

MECHANISM/PROPERTYMANAGER

operation()

in args

out args + return value

IDLSTUBS

Delegate

SysCond

SysCond

SysCond

IDLSKELETON

OBJECTADAPTER

ORB IIOP ORBIIOP

CLIENT OBJECT(SERVANT)OBJECT(SERVANT)

OBJREF

CLIENT

DelegateContract

SysCond

Contract

Network

MECHANISM/PROPERTYMANAGER

operation()

in args

out args + return value

IDLSTUBS

Delegate

SysCond

SysCond

SysCond

IDLSKELETON OBJECT

ADAPTER

ORB IIOP ORBIIOP

CLIENT OBJECT(SERVANT)OBJECT(SERVANT)

OBJREF

8 05/01/02 ISORC 2002

The QuO Toolkit Supports Building Adaptive Applications or Adding Adaptation to Existing Apps

• QuO aspect languages– Contract description language and

adaptive behavior description language

– Code generators that weave QuO code into Java and C++ applications

• System Condition Objects– Provide interfaces to resources,

managers, and mechanisms

• QuO Runtime Kernel– Contract evaluator– Factory object which instantiates

contract and system condition objects

• Instrumentation library• QuO gateway

– Insertion of special purpose transport layers and adaptation below the ORB

QuO GatewayQuO Gateway

IIOPGlue

Control

Clie

nt-S

ide

OR

B

IIOP Group Replication (AQuA)

WAN

Bandwidth Reservation (DIRM)

IIOP over TCP/IP (default)

IIOPGlue

Control

IIOP

Serv

er-S

ide

OR

B

CLIENT

DelegateContract

SysCond

Contract

Network

MECHANISM/PROPERTYMANAGER

operation()

in args

out args + return value

IDLSTUBS

Delegate

SysCond

SysCond

SysCond

IDLSKELETON OBJECT

ADAPTER

ORB IIOP ORBIIOP

CLIENT OBJECT(SERVANT)OBJECT(SERVANT)

OBJREF

CORBA IDL

CodeGenerators

CodeGenerators

Contract DescriptionLanguage (CDL)

Adaptation SpecificationLanguage (ASL)

QuO RuntimeQuO Runtime

Delegates Contracts

9 05/01/02 ISORC 2002

QuO Components Are Packaged into Reusable Bundles of “Systemic Behavior” Called Qoskets

• The Qosket encapsulates a set of contracts (CDL), system condition objects (IDL), and QoS adaptive behavior (ASL)

• The Qosket exposes interfaces to access QuO controls and information (specified in IDL)

• The Qosket separates the functional adaptive behavior (business logic) from the QoS adaptive behavior and the middleware controls from the QoS mechanisms

Qosket

Adapter Interface

Delegate Interface

Contracts SystemConditionObjects

CallbackObjects

QosketImplementation

HelperMethods

DelegateTemplates

10 05/01/02 ISORC 2002

Hand-written

Adapter-Qosket.id

lHand-written

Delegate-Qosket.id

l

Qosket-Impl.java

Contract.cdl

Hand-writtenSyscond-Impl.java

Hand-written Syscond.idl

Reusable Qosket

Callback-Impl.java

Callback.idl

Adapt-Template.asl

Hand-writtenimplements (java)

QDL compilergenerated

Exposed through an External Interface

QBAdapter.java

Hand-written

Hand-written Business.idl

Specialized to a Business Interface

Adapt.asl

Writing a Qosket, Instantiating and Wrapping It, and Specializing It to a Functional Interface

Hand-written

extends (java)

QBWrapper.java

QDL compilergenerated Delegate

extends (java)

QDL compilergenerated

QDL compilergenerated

QDL compilergenerated

Adapter-Qosket.java

Delegate-Qosket.java

AdapterQosket-Skel.java

implements(java)

implements (java)

extends (java)

Contract

IDL compilergenerated

Syscond StubCallback Stub

Key QDL or IDL code Native language code Functional QoS

11 05/01/02 ISORC 2002

An Example: A Video Distribution ApplicationControl Station Host 5

CORBA A/VStreaming Service

Host 1

MPEGFile

Host 4

Wired

WirelessEthernet

VideoDistributorProcess 1

VideoDistributorProcess 2

VideoDistributorProcess 3

VideoSourceProcess

Filter

FilterContract

Host 2

MPEGFile

VideoSourceProcess

Filter

FilterContract

Host 3Video

SourceProcess

Scale/Compress

QualityContract

BandwidthManagement

BandwidthManagement

BandwidthManagement

ThroughputContracts

Displays

Control Station Host 6

ThroughputContracts

Displays

Control Station Host 7

ATRContract

ATRDisplay

Uses off-the-shelf components• QuO adaptive middleware• Real-time CORBA ORB (TAO)

- Naming Service- A/V Streaming Service- AQoSA

• DVDViewer• Simulated ATR

Must maintain QoS Requirements, make Tradeoffs• Timeliness of the video• Fidelity of the video• Data quality of the video

Heterogeneity• Data formats - MPEG, PPM• Mechanisms

- RSVP, DiffServ- Filtering, scaling, compression

• Networking- Wired Ethernet- Wireless Ethernet

Qoskets are used to support the QoS properties• async_sender - threading and thread priorities• atr - controls image quality to support image processing• distributor - monitors and controls throughput• filter - filters frames• fragment - fragments and re-assembles frames to support rsvp• fragment_ppm - ditto for PPM• frame_proc - introduces tasking• header - creates and processes RTP headers for MPEG• rsvp - determines and controls RSVP policy

12 05/01/02 ISORC 2002

A Qosket That Measures Throughput and Reserves Bandwidth

Filter?

VideoDisplay

reservationstatus

A/VStreaming

Service

VideoDistributor

StreamEndpointDelegateBWReserveQosket

send_frame()

Sequence no.

reserve_bandwidth()

NormalDegraded

Unusable

requestreservation

send_frame()

Timestamp

AQoSA

RSVP

throughput

13 05/01/02 ISORC 2002

RSVP Fragmentation Problem Solved by Use of QuO Qoskets

• MPEG in UDP packets were being fragmented by IP.

• Routers did not know that fragmented IP packets were part of a RSVP reservation of UDP packets.

• Solution: QuO delegates for fragmentation/reassembly, so UDP packets are not fragmented at IP level.

FrameFilter

Delegate

TAO_AV_Protocol_Object

TAO_AV_UDP_Objecttransport->send()

distributor.{h,cpp}

Distributor Code

Receive FrameEvent Callback

wrapper-> send_frame()

Distributor

receiver.{h,cpp}

Receiver Code

Receive FrameEvent Callback

FragmentFrame

Delegate

ReassembleFrame

Delegate

Receiver

14 05/01/02 ISORC 2002

Support for Real-time Video Delivery Using Middleware Adaptation

0

5

10

15

20

25

30

35

0 30 60 90 120

150

180

210

240

270

300

Without adaptation

With adaptationLoad

Adaptation

No adaptation

Mean lateness Max lateness

5.400 sec 32.696 sec

0.067 sec 1.930 sec

Execution time (secs)

Add

ition

al la

tenc

y (s

ecs)

•Sender, distributor, and receiver running on 3 200 MHz PCs (Linux), 128 MB memory, TCP/IP

•Additional 60% CPU load introduced on second stage (3 processes requesting 20% load each) starting at approx. 60 secs, removed at approx. 120 secs.

Under load

15 05/01/02 ISORC 2002

0

100

200

300

400

500

600

700

0 100 200 300 400 500 600

Number of I frames sent

Nu

mb

er o

f I f

ram

es r

ecei

ved

No Adaptation

Frame dropping

Frame dropping andnetwork reservation

Load

Controlling Frame Loss Using QuO Adaptation

•Sender, distributor on a Pentium III 933 MHz, 512 MB RAM; Receiver on a Pentium II 200 MHz, 144 MB of RAM; all Linux, with 10 Mbps link, using UDP.

•Additional Network load introduced after 60 seconds, removed after 120 seconds

•Without middleware control, network layer will drop packets without application knowledge

•Frame dropping dropped the less important B and P frames in favor of the I frames

Frame Dropping

No adaptation

# I Frames Sent # Received

600 481

600 600

RSVP + FrameDropping

# Lost

600

119

0

600 0

16 05/01/02 ISORC 2002

Conclusions and Next Steps

• We have presented a component model, called Qoskets, that works with the QuO middleware, for encapsulating adaptive QoS behaviors

• More information on QuO, including qoskets, is available at http://www.dist-systems.bbn.com/tech/QuO

• QuO open-source software is at http://www.dist-systems.bbn.com/tech/QuO/release

• The example presented in this talk is at http://www.dist-systems.bbn.com/projects/AIRES/UAV

• Current, ongoing work– Comparison, coordination with CORBA Component Model (CCM)

– Composition of qoskets

– Coordinated RT CORBA, DiffServ for end-to-end RT behavior