metacomputing directory service (mds) a directory service for configuring high-performance...

Metacomputing Directory Service (MDS)

A Directory Service for Configuring High-Performance Distributed Computations (Fitzgerald et. al.)

Joel Thomas

Overview

Introduction Design Considerations Current Approaches Design of MDS Implementation MDS With Globus Future Conclusion

Introduction

High performance distributed computing requires selection, configuration

To make good decisions, timely, accurate information is invaluable

Currently, have nonstandard ways to store/access the right kind of data

The Metacomputing Directory Service (MDS) hopes to provide a scalable service to handle this need

Design Considerations

Performance Scalability and cost Uniformity Expressiveness Extensibility

Design Considerations(cont.)

Multiple Information Sources Dynamic data Flexible access Security Deployability Decentralized maintenance

Current Approaches

Uname, sysinfo Simple Network Management Protocol(SNMP),

NIS(Network Information Service) PVM(Parallel Virtual Machine), MPICH (Message

Passing Interface) DNS (Domain Name Service) X.500, LDAP (Lightweight Directory Access

Protocol) Chose to integrate existing systems as much as

possible But use LDAP to organize

Design of MDS

Representation and Data Access Data Model Implementation

Representation

Borrows straight from LDAP Entry: instance of person, network, computer, etc. Type is created by associating an object class Directory Information Tree (DIT)

hierarchical, tree-structured name space

Distinguished Name (DN) path from leaf to root entry <hn = dark.mcs.anl.gov

ou = MCS, o = Argonne National Laboratory, o = Globus, c = US >

DIT example

Object Classes

LDAP has set of standard class definitions, which can be extended (top, group classes, etc.)

Defines attributes for an entry, and what values those attributes may contain

Attributes may be required or optional Can use inheritance to build new classes

Data Model

Consists of DIT hierarchy and object class definitions

Computer centric, not network centric People, hosts, communication networks under

organization

Data Model (cont.)

GlobusHost GlobusNetwork

Link protocol (ATM, Ethernet) Network topology (bus, ring) Physical media (copper, fiber)

GlobusNetworkInterface Physical characteristics (speed) and hardware address

Hosts contain network interfaces Network interfaces attached to networks

Configuration example

MDS representation

Data Model (cont.)

Physical vs. Logical Network info Single network, multiple protocol stacks Each may have distinct interface and performance

Images: multiple logical views of the same physical network (RFC 1609) contains new information, and pointer back to physical Physical has reference to all images that point to it GlobusHostImage GlobusNetworkImage GlobustNetworkInterfaceImage

Implementation

While LDAP interface met most needs, default implementation was somewhat slow

Problems: Single information provider Client/server architecture Scope of Data

Allows info providers to provide information per attribute

Time to Live (TTL) for attribute (before refreshing) Update scope: process, computation, global

Sample queries

What are the network interfaces for ATM? Find GlobusNetworkInterface->GlobusNetwork

(link_protocol=ATM) For that GNI, GlobusNetworkInterface-

>GlobusNetworkInterfaceImage (ip_address)

Which nodes can use vendor protocols on fast internal networks? Two GlobusHostImages belong to the same

GlobusNetworkImage object

MDS Recap

MDS with Globus

Nexus communication module Uses MDS to find out what low-level mechanisms are

between modules Select between mechanism, based on rules Rules based on dynamic information or programmer

preference Code runs unchanged, but can adapt to meet

performance needs

I-WAY TestBed (heterogeneous virtual machines) Resource Location

Resource Brokers

Future

Use in GUSTO Extend Globus components to use MDS Expand information sources Optimize for common operations Sophisticated applications (e.g. resource

scheduling) Performance metrics

Conclusion

Designed to provide dynamic information in a timely, scalable manner

Take advantage of LDAP flexibility and growing usage in industry

Provide information richness in extensible manner