Next Generation Distributed Systems:
The and dynamicTAO approach
Fabio [email protected]
Department of Computer ScienceUniversity of São Paulo, Brazil
http://www.ime.usp.br/~kon
University of Tromsø 2
Introduction
Modern Computing Environments Hardware diversity: embedded systems, PDAs,
laptops, workstations, supercomputers.
Software diversity: different programming languages,
component architectures, operating systems.
Mobile computers
Mobile users (different accounts in different systems)
University of Tromsø 3
Highly-Dynamic Environments
Frequent changes:
1. Structural changes HW and SW upgrades, OS patches,
protocol updates
2. Dynamic changes availability of memory, CPU, and network
bandwidth; connectivity, physical location
University of Tromsø 4
Goal of Current Research
Facilitate management of dynamic, heterogeneous computing environments for: Users
System administrators
Developers
University of Tromsø 5
The Approach
“The Network is the Computer” , Sun microsystems.
Network-Centrism : user profiles, user environments services, applications, components
WYNIWYG: (What You Need Is What You Get) dynamic instantiation of applications and
services automatic configuration
University of Tromsø 6
From where can we start?
Run on multiple hardware platforms
Run on top of different OSes
Support different programming languages
Support dynamism, late binding, components
Solution: OMG IDL CORBA ORBs Standard CORBA Services (Naming, Trading,
Persistence)
University of Tromsø 7
But There Was A Problem
Conventional ORBs were static: Fixed threading model Fixed transport protocol: IIOP (over TCP/IP) Fixed security strategy Fixed scheduling
Inadequate for a wide range of applications: Multimedia Mobile Computing Adaptive Applications
University of Tromsø 8
Reflective ORB
Allows inspection and dynamic reconfiguration of the ORB internal engine.
1. dynamicTAO : an extension of the TAO ORB [Schmidt] very complete, big
2. LegORB (now, UIC ) : a component-based ORB not complete, but expanding very small (minimal client 6K or 20K, minimal server
30K)
3. OpenORB : [Blair et al], University Of Lancaster prototype in Python, implementation in C++/COM
University of Tromsø 9
What is missing?
We have: Reflective Middleware layer supporting distributed
objects in a dynamically configurable way. Standard services for Naming, Trading, Security,
Persistence, Transactions, Events.
We still need: Support for automatic configuration. Dynamic instantiation of user environments. Dynamic resource management.
University of Tromsø 10
2K Services
Component Repository Automatic Configuration Distributed Resource Management Mobile Configuration Agents
User Environment Service Distributed QoS Compilation Service Security, Data Management, ...
University of Tromsø 11
The 2K Architecture
University of Tromsø 12
Automatic Configuration Service
Automatically instantiates applications and services by assembling their components.
Based on Prerequisites : static representation of
dependencies.
ComponentConfigurators : dynamic
representation of dependencies.
University of Tromsø 13
Prerequisites
What a component needs to run: nature of hardware resources share of the hardware resources software services (i.e., components) it
requires
Video Client example: PC with Sound card 50% of CPU >300MHz CORBA Video Service
University of Tromsø 14
Automatic Configuration Process
1. Fetches component code and prerequisites from the Component Repository.
2. Dynamically link component code into the application address-space.
3. Based on the prerequisites, repeats the process for other components.
University of Tromsø 15
Automatic Configuration Architectural Framework
ComponentRepository
PrerequisiteParser
PrerequisiteResolver
QoS-AwareResource Manager Cache
load application
return reference
fetch prerequisites
fetch components
University of Tromsø 16
Component Configurators
Reify dynamic inter-component dependencies.
Created on-the-fly by the Prerequisite Resolver.
System and application software can inspect and reconfigure the Dependence Graph.
University of Tromsø 17
ComponentConfiguratorFramework
Allows browsing, inspection, and reconfiguration
Can be customized through inheritance
Clear separation of concerns
University of Tromsø 18
QoS-Aware Distributed Resource Management
Global Resource Manager (GRM) one in each cluster maintains an approximate view of the cluster
resource utilization
Local Resource Manager (LRM) runs in each node exports the state of the local resources Has a Real-Time Scheduler (DSRT)
admission control, reservation, and scheduling
University of Tromsø 19
Loading an Application with the Resource Management Service
1. Client contacts local LRM, giving application name and QoS requirements
2. LRM performs admission test
3. Request forwarded to GRM
4. GRM forwards request to best candidate
5. Remote LRM performs admission test, reservation, and runs AutoConfig.
University of Tromsø 20
AutoConfig ServiceLoading Several Components
University of Tromsø 21
The 2K Architecture
University of Tromsø 22
Dynamically Configurable Middleware:Reflective ORBs
Reflective Systems [Smith 84]
Meta-Object Protocol [Kiczales 91]
Reflective ORBs [Singhai and Campbell 97]
The ORB maintains a representation of its own internal structure, supporting Inspection Dynamic Reconfiguration Causal Connection
University of Tromsø 23
dynamicTAO
Built as an extension of the TAO ORB [Schmidt et al]
Written in C++ Modular design based on
object-oriented design patterns TAO already supported startup configuration;
configuration file specifies strategies for concurrency (threading model) request demultiplexing scheduling connection management
University of Tromsø 24
Adding Support for Dynamic Configuration
dynamicTAO exports an interface called DynamicConfigurator, supporting
1. Transfer of components across the distributed system
2. Loading and unloading components
3. Inspecting and modifying the configuration of the ORB (and of applications running on top it)
University of Tromsø 25
dynamicTAO Architecture
University of Tromsø 26
Reifying the ORB Structure
ComponentConfigurator framework
Stores inter-component dependencies
Allows browsing, inspection, and reconfiguration
Can be customized through inheritance
University of Tromsø 27
dynamicTAO Structure
University of Tromsø 28
DynamicConfiguratorIDL Interface
interface DynamicConfigurator
{
stringList list_categories ();
stringList list_implementations (in string categoryName);
stringList list_loaded_implementations ()
stringList list_hooks (in string componentName);
string get_hooked_comp (in string componentName,
in string hookName);
string get_comp_info (in string componentName);
. . .
University of Tromsø 29
Manage Component Implementations loaded in
memory
long load_implementation (in string categoryName,
in string impName,
in string params, ...);
void hook_implementation (in string loadedImpName,
in string componentName,
in string hookName);
void suspend_implementation (in string loadedImpName);
void resume_implementation (in string loadedImpName);
void remove_implementation (in string loadedImpName);
void configure_implementation (in string loadedImpName,
in string message);
University of Tromsø 30
Manage the ORB Persistent Component Repository
void upload_implementation (in string categoryName,
in string impName,
in implCode binCode);
void download_implementation (in string categoryName,
inout string impName,
out implCode binCode);
void delete_implementation (in string categoryName,
in string impName);
};
University of Tromsø 31
Example of Dynamic Configuration
1. myRemoteOrb->upload_implementation (“Security”, “superSAFE”, superSAFE_impl);
2. newSecurityStrategy = myRemoteOrb->load_implementation (“Security”, “superSAFE”);
3. oldSecurityStrategy = myRemoteOrb->get_hooked_comp (“dynamicTAO”, “Security_Strategy”);
4. myRemoteOrb->hook_implementation (newSecurityStrategy, “dynamicTAO”, “Security_Strategy”);
5. myRemoteOrb->remove_implementation (oldSecurityStrategy);
University of Tromsø 32
Consistency
Dynamic reconfiguration may break the consistency of the internal ORB engine.
Consistency must be ensured by the ORB developer and by the component developer.
Achieved by creating customized subclasses of the ComponentConfigurator class: TAOConfigurator Servant1Configurator MonitoringStrategyConfigurator ...
University of Tromsø 33
Implementing Reconfigurable ORB Components
Two major things to consider:1. Transferring the state from the old
component to the new component2. Making sure that no dangling references to
the old component remain
Must customize TAOConfigurator or strategy configurator (e.g.
ThreadPoolConfigurator)
University of Tromsø 34
Accessing the ORB Reconfiguration Interface
1. Local or remote code through IDL
2. Telnet
3. Java GUI
4. Reconfiguration Agents
University of Tromsø 35
DOCTORDynamic ORB Configuration Tool
University of Tromsø 36
Mobile Agents
A mobile agent visits a collection of ORBs.
In each ORB along its path, it can install new components on the disk, dynamically link new components, inspect the state and configuration of the
ORB and the applications on top of it, reconfigure ORBs and applications.
University of Tromsø 37
A Flexible Framework
Different NetworkBrokers support different agent flavors. For example: simple, lightweight, script-based agents
(carrying data and DCP commands only). powerful, heavyweight, Java-based agents
(carrying data, bytecode, and dynamic state, taking autonomous decisions).
Simple agents are suitable for PDAs, embedded systems.
University of Tromsø 38
Reconfiguration with Mobile Agents
SysAdmins use a GUI to build agents for reconfiguration inspection
GUI is used to1. Build distribution
graph2. Select
reconfiguration and inspection commands
3. Visualize results.
University of Tromsø 39
Security
SecureAgentBroker uses the GSS-API and supports Role-Based Access Control.
Agents are signed and transmitted via secure connections, using encryption.
RBAC is used in each ORB to decide which commands each agent is allowed to perform.
University of Tromsø 40
The SecureAgentBroker
University of Tromsø 41
Open Problems
Support for fault-tolerance: fault-recovery when part of the
reconfiguration process fails within a node fault-recovery when the reconfiguration
fails in part of the distributed system atomic transactions
Deploying agents for (re)configuration of active spaces in ubiquitous computing.
University of Tromsø 42
Applications of Reflective ORBs
Completed Prototypes: Flexible Object Monitoring Service Dynamic Security Service Multimedia applications (Nahrstedt, U. Illinois)
Ongoing work: Ubiquitous Computing (Illinois) Framework for Adaptive Applications (U. São
Paulo)
University of Tromsø 43
Monitoring Distributed Object Interactions
dynamicTAO shows how to adapt Applications also need to know when to
adapt
Monitoring Service: Can be dynamically loaded and unloaded No modifications in the applications Totally transparent to applications Uses the CORBA request-level interceptor [OMG98a]
University of Tromsø 44
Monitoring Service Architecture
University of Tromsø 45
Monitoring Service Overhead
String getHello (); Overhead:
when monitoring getHello: 10.1% with Monitoring Service on, but without
monitoring getHello: 2.0%
Ultra-2 Ultra-60
Client ServerFast Ethernet
University of Tromsø 46
Dynamic Security Service Prototype
Can be dynamically loaded and unloaded Uses
CORBA interceptors for access control Cherubim Security Framework [Campbell & Qian 98]
Java Active Capabilities flexible dynamic policies
implemented: DAC, MAC working on: RBAC, ABAC (?)
University of Tromsø 47
Open Problems
Improving Security Services how to provide security for millions of
distributed objects efficiently?
Monitoring Service tools: Specify what should be monitored Visualize monitored data graphically
University of Tromsø 48
The Future
As computing devices become pervasive in our society, we will encounter highly dynamic, heterogeneous
environments complex dependencies difficult management
We need standards and an integrated architecture to help manage this complexity in a clean and efficient way.
University of Tromsø 49
End of Part 1
Questions?
University of Tromsø 50
Security Architecture
Java Active Capabilities
Flexible Security Policies
Caching of Authorization Decisions
Auditing
University of Tromsø 51
Example of ConsistentDynamic Reconfiguration
Concurrency strategies1. Reactive (single-threaded)2. Thread-per-Connection3. Thread-Pool
Switching from 1 or 2 to any other: OK Switching from Thread-Pool:
problematic