Next Generation Distributed Systems:

The and dynamicTAO approach

Fabio Konkon@ime.usp.br

Department of Computer ScienceUniversity of São Paulo, Brazil

http://www.ime.usp.br/~kon

University of Tromsø 2

Introduction

Modern Computing Environments Hardware diversity: embedded systems, PDAs,

laptops, workstations, supercomputers.

Software diversity: different programming languages,

component architectures, operating systems.

Mobile computers

Mobile users (different accounts in different systems)

University of Tromsø 3

Highly-Dynamic Environments

Frequent changes:

1. Structural changes HW and SW upgrades, OS patches,

protocol updates

2. Dynamic changes availability of memory, CPU, and network

bandwidth; connectivity, physical location

University of Tromsø 4

Goal of Current Research

Facilitate management of dynamic, heterogeneous computing environments for: Users

System administrators

Developers

University of Tromsø 5

The Approach

“The Network is the Computer” , Sun microsystems.

Network-Centrism : user profiles, user environments services, applications, components

WYNIWYG: (What You Need Is What You Get) dynamic instantiation of applications and

services automatic configuration

University of Tromsø 6

From where can we start?

Run on multiple hardware platforms

Run on top of different OSes

Support different programming languages

Support dynamism, late binding, components

Solution: OMG IDL CORBA ORBs Standard CORBA Services (Naming, Trading,

Persistence)

University of Tromsø 7

But There Was A Problem

Conventional ORBs were static: Fixed threading model Fixed transport protocol: IIOP (over TCP/IP) Fixed security strategy Fixed scheduling

Inadequate for a wide range of applications: Multimedia Mobile Computing Adaptive Applications

University of Tromsø 8

Reflective ORB

Allows inspection and dynamic reconfiguration of the ORB internal engine.

1. dynamicTAO : an extension of the TAO ORB [Schmidt] very complete, big

2. LegORB (now, UIC ) : a component-based ORB not complete, but expanding very small (minimal client 6K or 20K, minimal server

30K)

3. OpenORB : [Blair et al], University Of Lancaster prototype in Python, implementation in C++/COM

University of Tromsø 9

What is missing?

We have: Reflective Middleware layer supporting distributed

objects in a dynamically configurable way. Standard services for Naming, Trading, Security,

Persistence, Transactions, Events.

We still need: Support for automatic configuration. Dynamic instantiation of user environments. Dynamic resource management.

University of Tromsø 10

2K Services

Component Repository Automatic Configuration Distributed Resource Management Mobile Configuration Agents

User Environment Service Distributed QoS Compilation Service Security, Data Management, ...

University of Tromsø 11

The 2K Architecture

University of Tromsø 12

Automatic Configuration Service

Automatically instantiates applications and services by assembling their components.

Based on Prerequisites : static representation of

dependencies.

ComponentConfigurators : dynamic

representation of dependencies.

University of Tromsø 13

Prerequisites

What a component needs to run: nature of hardware resources share of the hardware resources software services (i.e., components) it

requires

Video Client example: PC with Sound card 50% of CPU >300MHz CORBA Video Service

University of Tromsø 14

Automatic Configuration Process

1. Fetches component code and prerequisites from the Component Repository.

2. Dynamically link component code into the application address-space.

3. Based on the prerequisites, repeats the process for other components.

University of Tromsø 15

Automatic Configuration Architectural Framework

ComponentRepository

PrerequisiteParser

PrerequisiteResolver

QoS-AwareResource Manager Cache

load application

return reference

fetch prerequisites

fetch components

University of Tromsø 16

Component Configurators

Reify dynamic inter-component dependencies.

Created on-the-fly by the Prerequisite Resolver.

System and application software can inspect and reconfigure the Dependence Graph.

University of Tromsø 17

ComponentConfiguratorFramework

Allows browsing, inspection, and reconfiguration

Can be customized through inheritance

Clear separation of concerns

University of Tromsø 18

QoS-Aware Distributed Resource Management

Global Resource Manager (GRM) one in each cluster maintains an approximate view of the cluster

resource utilization

Local Resource Manager (LRM) runs in each node exports the state of the local resources Has a Real-Time Scheduler (DSRT)

admission control, reservation, and scheduling

University of Tromsø 19

Loading an Application with the Resource Management Service

1. Client contacts local LRM, giving application name and QoS requirements

2. LRM performs admission test

3. Request forwarded to GRM

4. GRM forwards request to best candidate

5. Remote LRM performs admission test, reservation, and runs AutoConfig.

University of Tromsø 20

AutoConfig ServiceLoading Several Components

University of Tromsø 21

The 2K Architecture

University of Tromsø 22

Dynamically Configurable Middleware:Reflective ORBs

Reflective Systems [Smith 84]

Meta-Object Protocol [Kiczales 91]

Reflective ORBs [Singhai and Campbell 97]

The ORB maintains a representation of its own internal structure, supporting Inspection Dynamic Reconfiguration Causal Connection

University of Tromsø 23

dynamicTAO

Built as an extension of the TAO ORB [Schmidt et al]

Written in C++ Modular design based on

object-oriented design patterns TAO already supported startup configuration;

configuration file specifies strategies for concurrency (threading model) request demultiplexing scheduling connection management

University of Tromsø 24

Adding Support for Dynamic Configuration

dynamicTAO exports an interface called DynamicConfigurator, supporting

1. Transfer of components across the distributed system

2. Loading and unloading components

3. Inspecting and modifying the configuration of the ORB (and of applications running on top it)

University of Tromsø 25

dynamicTAO Architecture

University of Tromsø 26

Reifying the ORB Structure

ComponentConfigurator framework

Stores inter-component dependencies

Allows browsing, inspection, and reconfiguration

Can be customized through inheritance

University of Tromsø 27

dynamicTAO Structure

University of Tromsø 28

DynamicConfiguratorIDL Interface

interface DynamicConfigurator

{

stringList list_categories ();

stringList list_implementations (in string categoryName);

stringList list_loaded_implementations ()

stringList list_hooks (in string componentName);

string get_hooked_comp (in string componentName,

in string hookName);

string get_comp_info (in string componentName);

. . .

University of Tromsø 29

Manage Component Implementations loaded in

memory

long load_implementation (in string categoryName,

in string impName,

in string params, ...);

void hook_implementation (in string loadedImpName,

in string componentName,

in string hookName);

void suspend_implementation (in string loadedImpName);

void resume_implementation (in string loadedImpName);

void remove_implementation (in string loadedImpName);

void configure_implementation (in string loadedImpName,

in string message);

University of Tromsø 30

Manage the ORB Persistent Component Repository

void upload_implementation (in string categoryName,

in string impName,

in implCode binCode);

void download_implementation (in string categoryName,

inout string impName,

out implCode binCode);

void delete_implementation (in string categoryName,

in string impName);

};

University of Tromsø 31

Example of Dynamic Configuration

1. myRemoteOrb->upload_implementation (“Security”, “superSAFE”, superSAFE_impl);

2. newSecurityStrategy = myRemoteOrb->load_implementation (“Security”, “superSAFE”);

3. oldSecurityStrategy = myRemoteOrb->get_hooked_comp (“dynamicTAO”, “Security_Strategy”);

4. myRemoteOrb->hook_implementation (newSecurityStrategy, “dynamicTAO”, “Security_Strategy”);

5. myRemoteOrb->remove_implementation (oldSecurityStrategy);

University of Tromsø 32

Consistency

Dynamic reconfiguration may break the consistency of the internal ORB engine.

Consistency must be ensured by the ORB developer and by the component developer.

Achieved by creating customized subclasses of the ComponentConfigurator class: TAOConfigurator Servant1Configurator MonitoringStrategyConfigurator ...

University of Tromsø 33

Implementing Reconfigurable ORB Components

Two major things to consider:1. Transferring the state from the old

component to the new component2. Making sure that no dangling references to

the old component remain

Must customize TAOConfigurator or strategy configurator (e.g.

ThreadPoolConfigurator)

University of Tromsø 34

Accessing the ORB Reconfiguration Interface

1. Local or remote code through IDL

2. Telnet

3. Java GUI

4. Reconfiguration Agents

University of Tromsø 35

DOCTORDynamic ORB Configuration Tool

University of Tromsø 36

Mobile Agents

A mobile agent visits a collection of ORBs.

In each ORB along its path, it can install new components on the disk, dynamically link new components, inspect the state and configuration of the

ORB and the applications on top of it, reconfigure ORBs and applications.

University of Tromsø 37

A Flexible Framework

Different NetworkBrokers support different agent flavors. For example: simple, lightweight, script-based agents

(carrying data and DCP commands only). powerful, heavyweight, Java-based agents

(carrying data, bytecode, and dynamic state, taking autonomous decisions).

Simple agents are suitable for PDAs, embedded systems.

University of Tromsø 38

Reconfiguration with Mobile Agents

SysAdmins use a GUI to build agents for reconfiguration inspection

GUI is used to1. Build distribution

graph2. Select

reconfiguration and inspection commands

3. Visualize results.

University of Tromsø 39

Security

SecureAgentBroker uses the GSS-API and supports Role-Based Access Control.

Agents are signed and transmitted via secure connections, using encryption.

RBAC is used in each ORB to decide which commands each agent is allowed to perform.

University of Tromsø 40

The SecureAgentBroker

University of Tromsø 41

Open Problems

Support for fault-tolerance: fault-recovery when part of the

reconfiguration process fails within a node fault-recovery when the reconfiguration

fails in part of the distributed system atomic transactions

Deploying agents for (re)configuration of active spaces in ubiquitous computing.

University of Tromsø 42

Applications of Reflective ORBs

Completed Prototypes: Flexible Object Monitoring Service Dynamic Security Service Multimedia applications (Nahrstedt, U. Illinois)

Ongoing work: Ubiquitous Computing (Illinois) Framework for Adaptive Applications (U. São

Paulo)

University of Tromsø 43

Monitoring Distributed Object Interactions

dynamicTAO shows how to adapt Applications also need to know when to

adapt

Monitoring Service: Can be dynamically loaded and unloaded No modifications in the applications Totally transparent to applications Uses the CORBA request-level interceptor [OMG98a]

University of Tromsø 44

Monitoring Service Architecture

University of Tromsø 45

Monitoring Service Overhead

String getHello (); Overhead:

when monitoring getHello: 10.1% with Monitoring Service on, but without

monitoring getHello: 2.0%

Ultra-2 Ultra-60

Client ServerFast Ethernet

University of Tromsø 46

Dynamic Security Service Prototype

Can be dynamically loaded and unloaded Uses

CORBA interceptors for access control Cherubim Security Framework [Campbell & Qian 98]

Java Active Capabilities flexible dynamic policies

implemented: DAC, MAC working on: RBAC, ABAC (?)

University of Tromsø 47

Open Problems

Improving Security Services how to provide security for millions of

distributed objects efficiently?

Monitoring Service tools: Specify what should be monitored Visualize monitored data graphically

University of Tromsø 48

The Future

As computing devices become pervasive in our society, we will encounter highly dynamic, heterogeneous

environments complex dependencies difficult management

We need standards and an integrated architecture to help manage this complexity in a clean and efficient way.

University of Tromsø 49

End of Part 1

Questions?

University of Tromsø 50

Security Architecture

Java Active Capabilities

Flexible Security Policies

Caching of Authorization Decisions

Auditing

University of Tromsø 51

Example of ConsistentDynamic Reconfiguration

Concurrency strategies1. Reactive (single-threaded)2. Thread-per-Connection3. Thread-Pool

Switching from 1 or 2 to any other: OK Switching from Thread-Pool:

problematic