December 16th, 2002
Composable Distributed Real-time Systems
Krishnakumar [email protected]
Department Of Computer ScienceWashington University, St.Louis, MO
Advisor: Dr. Christopher D. GillFunded by DARPA PCES under Contract to
Boeing (F33615-00-C-3048) &Washington University in St.Louis (F33615-00-C-
1697)
2
Krishnakumar B Composable Distributed Real-Time Systems
Washington University in St.Louis
Outline of Talk• Components, Component
Models• Middleware Composition
Challenges• Solutions• Future work• Questions?
InterfaceRepository
IDLCompiler
ImplementationRepository
ClientOBJREF
Object(Servant)
in argsoperation()out args +
return
DIIIDL
STUBSORB
INTERFACE
IDLSKEL
DSI
Object Adapter
ORB CORE GIOP/IIOP/ESIOPS
3
Krishnakumar B Composable Distributed Real-Time Systems
Washington University in St.Louis
Project Synopsis• Context
– Developing distributed real-time systems using components
• Problems– Configuration information tied to
implementation– Context sensitive QoS
specification– No validation of QoS properties– Ad hoc deployment
• Solution Approach– Specify configuration as meta-
data – Decouple QoS specification from
context dependencies– Validate configuration– Standardize deployment
Packaging & Deployment
Component Properties
Display
Database
FileComponent Assembly
Database
Display
File
File Properties
Database Properties
4
Krishnakumar B Composable Distributed Real-Time Systems
Washington University in St.Louis
Overview of Components• Component
– Encapsulated part of a software system
– Implements a specific service or set of services
• Components vs. Objects– Multiple views per
component – Transparent navigation
between views– Extensibility without
inheritance– Higher-level Execution
Environment
FileComponent
FILE*File
UnixFile
FileStream
FileConnector
Different Views of aComponent
Client A
Client B
Client C
Client D
Navigation
5
Krishnakumar B Composable Distributed Real-Time Systems
Washington University in St.Louis
Properties of Component Models• Component types, properties, inter-connections, interfaces• Component run-time infrastructure• Component Identity• Underlying Object Model
Features EnterpriseJavaBeans
Microsoft.NET
CORBAComponent
Model
Basic Type JavaBean .NETComponent
CORBAComponent
Transport RMI/IIOP SOAP IIOP
Object Model JVM CLR CORBA
Component Identification
JNDI GUID UUID
Unit of deployment JAR Files .NET Assembly
SoftPkg
6
Krishnakumar B Composable Distributed Real-Time Systems
Washington University in St.Louis
CORBA Component Model• Component Model
specification from OMG• Builds on strengths of
CORBA• Key elements
– Component• Facets• Receptacles• Event Sources• Event Sinks
– ComponentHome– Container– Component Server– ORB Services
Facets
ComponentServer
Container
ComponentHome
ORB Middleware
Operating System
Transaction Persistence Events Security QoS
Component
Container
ComponentHome
Component Component ComponentReceptacle
External Interfaces
7
Krishnakumar B Composable Distributed Real-Time Systems
Washington University in St.Louis
Component Integrated ACE ORB (CIAO)
• CCM implementation based on TAO
• QoS-aware Containers• Introspection interfaces• QoS assurance
mechanisms • Separate developer roles
– Application developer– Component
developer– QoS provisioners– ORB configurators
• Specify QoS as metadata
Client Application Server
Component Assembly
RT-ORB
in args
out args + return value
Operation ()
QoSMechanism
Plug ins
QoSMechanism
Plug ins
ClientConfiguration
Aggregate
QoSAdaptation
Container
CORBAComponent
ComponentHome
Real-time POA
QoS PropertyAdaptor
QoS Policies
Re
flect
QoSAdaptation
QoSAdaptation
QoSMechanism
Plug ins
NamedPolicy
Aggregate
NamedPolicy
Aggregate
ObjectReference
QoSAdaptation
QoSMechanism
Plug ins
QoS Policies
Component ConnectionSpecifications
Component & Home Impls
Deployment&
ConfigurationMechanism
PackagingTool
Assembly
ToolComponentAssemblyPackage
Home Properties Component Properties
Deployment
Tool
AssemblyDescriptor
CORBAComponent
Package
softpkgDescriptor
CORBAComponentPackage
Component Interconnection
Definitions
CORBAComponentPackage
8
Krishnakumar B Composable Distributed Real-Time Systems
Washington University in St.Louis
Motivating Application• CORBA Event Channel
– Typical interactions:• I/O arrives• Proxies demarshal
data• Facades process data
• Problems– Numerous
configuration options– Configuration coupled
with implementation– No configuration
validation• Potential run-time
failure
9
Krishnakumar B Composable Distributed Real-Time Systems
Washington University in St.Louis
Problems & SolutionsProblem Solution Contribution
Configuration tied with functionality
Specify configuration information as meta-data
Configuration decoupled from functionality
Implicit dependencies of QoS properties on context
Make context explicit as part of packaging
Well-defined dependencies
Invalid configuration resulting in run-time failure
Validate configuration meta-data prior to deployment
Static validation of configuration
Ad hoc manual deployment
Define standard mechanisms for deployment
Automated deployment
10
Krishnakumar B Composable Distributed Real-Time Systems
Washington University in St.Louis
Configuration of Components• Context
– Configuration of distributed real-time systems
• Problem– Overwhelming options– Configuration tangled
with functionality• Strong Coupling
– Ad hoc specification• Interoperability
problems
– Premature finalization• Integration problems
ECDispatching ECDispatchingThreads
ECFiltering
ECSupplierFiltering
ECTimeout
ECScheduling
ECProxyPushConsumerCollection
ECProxySupplierLock
ECConsumerControl
ECSupplierControl
ECConsumerControlPeriod
ECSupplierControlPeriod
ECProxyConsumerLock
EventChannel
11
Krishnakumar B Composable Distributed Real-Time Systems
Washington University in St.Louis
Solution: Configuration as Meta-data• Meta-data
– Describes systemic properties
• Describe configuration as meta-data
• Separate configuration from functionality
• Use meta-language to describe meta-data– XML provides a basis– Ample scope for
• Customization • Extension
– Increased interoperability
ECDispatching ECDispatchingThreads
ECFiltering
ECSupplierFiltering
ECTimeout ECScheduling
ECProxyPushConsumerCollection
ECConsumerControl ECSupplierControl
ECConsumerControlPeriod ECSupplierControlPeriod
ECProxySupplierLockECProxyConsumerLock
XML !XML !
EventChannel
12
Krishnakumar B Composable Distributed Real-Time Systems
Washington University in St.Louis
Solution Applied in CIAO• CIAO uses ACEXML library• Based on Simple API for XML
(SAX) v2.0– Callback-based API– Low memory footprint
• Document Type Definition (DTD)
• Validating Parser• Features
– DTD Validator– XML Namespaces– Locator– Filters
• Extensible– Pluggable Content
Handlers
Content Handler
ACEXML Parser
Client2: create
Default Handler
1: extend
4: register
3: instantiate
5: read
<?xml version="1.0"?> <doc> <para>Hello, world!</para>
</doc>
6: callback
XML File
13
Krishnakumar B Composable Distributed Real-Time Systems
Washington University in St.Louis
Lessons Learned• Specifying configuration as meta-data
decouples– Configuration information from functionality– Act of configuration from functionality
• XML is very useful for describing meta-data– Customizable– Location transparent
• Allows out-of-band fetching of configuration information
• Interoperable
14
Krishnakumar B Composable Distributed Real-Time Systems
Washington University in St.Louis
Problems & SolutionsProblem Solution Contribution
Configuration tied with functionality
Specify configuration information as meta-data
Configuration decoupled from functionality
Implicit dependencies of QoS properties on context
Make context explicit as part of packaging
Well-defined dependencies
Invalid configuration resulting in run-time failure
Validate configuration meta-data prior to deployment
Static validation of configuration
Ad hoc manual deployment
Define standard mechanisms for deployment
Automated deployment
15
Krishnakumar B Composable Distributed Real-Time Systems
Washington University in St.Louis
Context Aware QoS Specification• Context
– Designing Components
• Multiple QoS assurance levels
• Different underlying middleware
• Problem– Developers exposed
to• Different QoS
features• Different
implementations
– Dependencies implicit
Vendor A’s ORB
Vendor B’s ORB
Thread Pool dispatch
Reactive dispatch
Component
16
Krishnakumar B Composable Distributed Real-Time Systems
Washington University in St.Louis
Solution: Package Dependencies• Identify dependencies
– External– Internal
• Make dependencies explicit
• Describe dependencies extensibly– Scope for
standardization– Third-party
integration• Check dependencies
during component installation
Component A
ORB AComponent B
ORB B
libEvent.solibNotify.so
libSecurity.so libPersistence.so
<<depends>> <<depends>>
<<depends>>
<<depends>>
<<depends>>
<<depends>>
<<depends>> <<depends>>
<<depends>>
17
Krishnakumar B Composable Distributed Real-Time Systems
Washington University in St.Louis
Solution Applied in CIAO (1/2)• CIAO captures dependencies using XML Software Package descriptors• Specification at multiple levels of granularity
– Individual implementation of same component – Component level– Package Level
<softpkg name="EventChannelPkg" version="0.1"> <pkgtype>CORBA Component</pkgtype> <title>EventChannel</title> <description>A simple Event Channel test </description> <dependency type="DSO" ><localfile name="libTAO.so"/> </dependency> <implementation id="EventChannel" > <os name="Linux" version="4,0,0,0" /> <processor name="x86" /> <compiler name="gcc" /> <programminglanguage name="C++" /> <propertyfile> <fileinarchive name="eventchannel.cpf"/></propertyfile> <code type="DSO" > <fileinarchive name="libService.so"/> <entrypoint>createEventChannel</entrypoint> </code> </implementation></softpkg>
18
Krishnakumar B Composable Distributed Real-Time Systems
Washington University in St.Louis
Solution Applied in CIAO (2/2)• META-INF
– Well-known descriptor file
• ACEXML parses META-INF• CIAO
– Uses Pluggable Handlers features of ACEXML
– Implements SoftPkgHandler
• Dependencies checked during installation
SoftPkgHandler
ACEXML Parser
CIAO
2: create
DefaultHandler
1: extend
4: register
5: read
<softpkg> <implementation id="EventChannel"> <code type="DSO" > <fileinarchive name="libService.so"/> </code> </implementation></softpkg>
6: callback
META-INF
ACEXML Validator
3: instantiate
19
Krishnakumar B Composable Distributed Real-Time Systems
Washington University in St.Louis
Lessons Learned• Dependency checking reduces run-time
surprises– Missing files within the package– Local environment files needed
• Standard format needed for interoperability– Different ORBs may use different methods to
specify• Dispatching models Thread pools, Reactive• Concurrency mechanisms Mutex, Semaphores• Different levels of filtering events
20
Krishnakumar B Composable Distributed Real-Time Systems
Washington University in St.Louis
Problems & SolutionsProblem Solution Contribution
Configuration tied with functionality
Specify configuration information as meta-data
Configuration decoupled from functionality
Implicit dependencies of QoS properties on context
Make context explicit as part of packaging
Well-defined dependencies
Invalid configuration resulting in run-time failure
Validate configuration meta-data prior to deployment
Static validation of configuration
Ad hoc manual deployment
Define standard mechanisms for deployment
Automated deployment
21
Krishnakumar B Composable Distributed Real-Time Systems
Washington University in St.Louis
Component Misconfiguration• Context
– Integration of application from COTS components
• Problem– Large number of
components• Check individually• Check overall system
– Manual integration • Error-prone• Doesn’t scale
ECConsumerControl ECConsumerControlPeriod
EventChannel
ECDispatching ECDispatchingThreads
== 0== “mt”
== “reactive” == 1
22
Krishnakumar B Composable Distributed Real-Time Systems
Washington University in St.Louis
Solution: Configuration Validation• Validate component
configuations– XML allows
specification of Document Type Definitions (DTD)
– Check configuration to ensure conformance to DTD
• QoS configuration validation should be done for– Individual components– Groups of components– Application wide in an
end-to-end fashion
ECConsumerControl ECConsumerControlPeriod
EventChannel
ECDispatching ECDispatchingThreads
== “mt”
== “reactive”
>= 1
>=1000
23
Krishnakumar B Composable Distributed Real-Time Systems
Washington University in St.Louis
PropertyFile
Solution Applied in CIAO• Component Property Files (cpf)
– Supports specifying simple types, structs, sequences, valuetypes
– Set of valid configurations • Validation done at deployment
– Minimize run-time failure• Minimize run-time exceptions• Allow static QoS provisioning<?xml version="1.0" encoding="UTF-8" ?><!DOCTYPE properties SYSTEM "properties.dtd"><properties> <struct name="ECProperties" type=“EC_Property"> <description> Configuration for a Simple Event Channel</description> <simple name="ECDispatching" type="string"> <value>mt</value> </simple> <simple name="ECDispatchingThreads" type=“long"> <value>2</value> </simple> <simple name="ECProxyConsumerLock" type="string"> <value>basic</value></simple> <simple name="ECConsumerControl" type="string"> <value>reactive</value></simple> <simple name="ECConsumerControlPeriod" type="long"> <value>50000</value> </simple> </struct></properties>
EC_Property
ECDispatching
ECDispatchingThreads
ECConsumerControl
ECConsumerControlPeriod
ECProxyConsumerLock
ACEXML Validator
SyntacticValidation
Configuration Validation
24
Krishnakumar B Composable Distributed Real-Time Systems
Washington University in St.Louis
Lessons Learned• How to parse Hedge Automata
– Hedge is a sequence of trees– An XML document is a “Hedge”
• Configuration validation helps reduce “pilot error”
• Currently ORB configuration is intricately tied to the Service Configurator– Any configurable entity in the ORB is a
ACE_Service_Object– Only way to locate entities is from the
Service_Repository• Properties parsed from XML have to converted
back to text to be re-parsed by Service Configurator
25
Krishnakumar B Composable Distributed Real-Time Systems
Washington University in St.Louis
Problems & SolutionsProblem Solution Contribution
Configuration tied with functionality
Specify configuration information as meta-data
Configuration decoupled from functionality
Implicit dependencies of QoS properties on context
Make context explicit as part of packaging
Well-defined dependencies
Invalid configuration resulting in run-time failure
Validate configuration meta-data prior to deployment
Static validation of configuration
Ad hoc manual deployment
Define standard mechanisms for deployment
Automated deployment
26
Krishnakumar B Composable Distributed Real-Time Systems
Washington University in St.Louis
Deployment• Context
– Deploying an application built using COTS components
• Problem– Complex applications
• Large no. of components
– Micro-management of components
• Difficulty in reasoning complete end-to-end behaviour
– Manual deployment inherently error-prone
– Ad hoc scripts no better
QoSProperties
Checksum
Version Dependencies
List of FilesQoS Specs ComponentInterconnections
ImplementationDLL DLL
QoSProperties
Checksum
Version Dependencies
ImplementationDLL DLL
27
Krishnakumar B Composable Distributed Real-Time Systems
Washington University in St.Louis
Solution : Component Packages• Packaging
– Logical grouping – Software Package
• Single Component
– Component Assembly• Multiple components
– Groups components with
• Implementation• Meta-data• Dependencies
• Packages are deployed as compressed (.ZIP) files
Meta-data
Component Component
Meta-data
Implementation
QoSProperties
Checksum
Version Dependencies
DLL DLL
List of FilesQoS Specs ComponentInterconnections
Assembly
Component Server
Deployment Application
Component Server Component Server
Assembly Assembly
ImplementationDLL DLL
Meta-data
QoSProperties
Checksum
Version Dependencies
28
Krishnakumar B Composable Distributed Real-Time Systems
Washington University in St.Louis
Package Descriptors (1/2)• Package descriptors• CORBA Component Descriptor (.ccd)
– Describes a single component• Interfaces, ports, QoS policies• QoS properties described via (.cpf) files
• CORBA Software Descriptor (.csd)– Describes implementations of a single component– Useful in deploying a single component
<corbacomponent> <componentrepid repid="IDL:BookStore:1.0" /> <componentkind> <entity> <servant lifetime="component" /> </entity> </componentkind> <security rightsfamily="corba" rightscombinator="secallrights"> </security> <threading policy="multithread" /> <componentfeatures name="BookStore" repid="IDL:BookStore:1.0"> <inheritscomponent repid="IDL:Acme/Store:1.0" /> <ports> <provides providesname="book_search" repid="IDL:BookSearch:1.0" facettag="1"> <uses usesname="fedex_rates" repid="IDL:ShippingRates:1.0" /> <emits emitsname="low_stock" eventtype="StockRecord"><eventpolicy policy="normal" /></emits> <publishes publishesname="offer_alert" eventtype="SpecialOffer"> <eventpolicy policy="normal" /> </publishes> </ports> </componentfeatures></corbacomponent>
29
Krishnakumar B Composable Distributed Real-Time Systems
Washington University in St.Louis
Package Descriptors (2/2)• CORBA Assembly descriptor (.cad)
– Describes a collection of components & their properties– Describes partitioning of the group of components– Describes dependencies among components
<componentassembly id="ZZZ123"> <description>Example assembly"</description> <componentfiles> <componentfile id="A"> <fileinarchive name="ca.ccsd"/> </componentfile> <componentfile id="B"> <fileinarchive name="cb.ccsd"/> </componentfile> </componentfiles> <partitioning> <homeplacement id="AaaHome"> <usagename>Example home for A components</usagename> <componentfileref idref="A"/> <componentimplref idref="an A impl"/> <homeproperties> <fileinarchive name="AHomeProperties.cpf"/> </homeproperties> <componentproperties> <fileinarchive name="defaultAProperties.cpf"/> </componentproperties> </homeplacement> </partitioning></componentassembly>
30
Krishnakumar B Composable Distributed Real-Time Systems
Washington University in St.Louis
Deployment Hierarchy• Packages are read by
Deployment agent• Descriptor of package is
read from META-INF• Unpacking of packages in
safe directory– Avoid trojan
implementations• Action determined by
contents of META-INF • Set of Helper classes
– ComponentInstallation– AssemblyFactory– Assembly– ServerActivator
• Used in bootstrapping deployment
AssemblyFactory Assembly
Deployment Application
ServerActivator
ComponentInstallation
ComponentServer
Container
CCMHome
CCMObject
<<instantiates>>
<<instantiates>>
<<instantiates>>
<<instantiates>>
<<instantiates>>
31
Krishnakumar B Composable Distributed Real-Time Systems
Washington University in St.Louis
Lessons Learned• Packaging makes component management
easier– Provides logical view of application subsytems– Aggregates groups of components– Provides COTS components in the literal sense
• Standardizing deployment– Solves problem once– Allows sharing of components– Reduce footprint
• Lazy component loading using Virtual Component design pattern
32
Krishnakumar B Composable Distributed Real-Time Systems
Washington University in St.Louis
Future Work• Complete integration of
Packaging & Deployment framework with CIAO
• Decouple ORB configuration from Service Configurator
• Configuration of ORB from structs and sequences
• Allow imposing policies in a hierarchical fashion
• Measure overhead• Increase user-friendliness of
the Deployment framework
Client Application Server
Component Assembly
RT-ORB
in args
out args + return value
Operation ()
QoSMechanism
Plug ins
QoSMechanism
Plug ins
ClientConfiguration
Aggregate
QoSAdaptation
Container
CORBAComponent
ComponentHome
Real-time POA
QoS PropertyAdaptor
QoS Policies
Re
flect
QoSAdaptation
QoSAdaptation
QoSMechanism
Plug ins
NamedPolicy
Aggregate
NamedPolicy
Aggregate
ObjectReference
QoSAdaptation
QoSMechanism
Plug ins
QoS Policies
Component ConnectionSpecifications
Component & Home Impls
Deployment&
ConfigurationMechanism
33
Krishnakumar B Composable Distributed Real-Time Systems
Washington University in St.Louis
Timeline
5/02
1 Apr, 20021 Apr, 2002
15 Jan, 200315 Jan, 2003
6/02 7/02 8/02 9/02 10/02 11/02 12/02 1/03
4/02XML Parser
11/6URL fetcher
6/02Namespace support
9/02Exception infrastructure
23/9Obstack improvements
10/02Exception Support
10/02XML Validator
10/02Component Packages
10/02QoS Specif ications
11/02Deployment Infrastructure
12/02Defense
1/03Integration
Non-software artifact
To be implemented
Implementation complete
Implemented byNanbor Wang
8/02WORDS 2003 paper
Component Packages refers to:1. Ability to package components as assemblies.2. Support for reading and modifying assemblies.
Deployment Infrastructure refers to:1. Ability to install a component(s) as assemblies.2. Setting up and tear down of a componenthierarchy using assemblies.
12/02Configuration Validation
34
Krishnakumar B Composable Distributed Real-Time Systems
Washington University in St.Louis
Concluding Remarks• Summary Of Benefits
– Specify configuration as meta-data – Decouple QoS specification from context dependencies– Validate configurations– Standardize deployment
• Programming effort – ACEXML
• Common infrastructure - 8K lines• Parser Enhancements - 3K lines• Validator - 6K lines and growing• Available under $ACE_ROOT/ACEXML
– Packaging & Deployment • 4K lines • Available under $ACE_ROOT/ACEXML/compass
• Availability– cvs co –r Validator ACE_wrappers/ACEXML– cvs co –r CIAO ACE_wrappers
• First release early 2003
35
Krishnakumar B Composable Distributed Real-Time Systems
Washington University in St.Louis
Acknowledgements• Dr. Chris Gill• Dr. Ron Cytron• Dr. David Levine• Dr. Doug Schmidt• All past and present members of the DOC
group