grid computing and the globus toolkit jennifer m. schopf argonne national lab

Grid Computing and the Grid Computing and the Globus ToolkitGlobus Toolkit

Jennifer M. SchopfJennifer M. Schopf

Argonne National LabArgonne National Lab

July 29, 2003 Grid Computing and Globus, Jennifer M. Schopf 2

Questions for you-Questions for you-

How many people know what Grids and Grid computing are?

How many people are familiar with Globus (GT2/GT3)?

How many have heard of OGSA/OGSI?


This talkThis talk

What is Grid Computing? Who’s using Grids? What is Globus? What does Globus do? Some other resources


What is a Grid?What is a Grid?

Shared resources Coordinated problem solving Multiple sites (multiple institutions)


Not A New IdeaNot A New Idea

Late 70’s – Networked operating systems Late 80’s – Distributed operating system Early 90’s – Heterogeneous computing Mid 90’s - Metacomputing

Then the “Grid” – Foster and Keselman, 1999


Broader ContextBroader Context

“Grid Computing” has much in common with major industrial thrusts– Business-to-business, Peer-to-peer, Application Service

Providers, Storage Service Providers, Distributed Computing, Internet Computing…

Sharing issues not adequately addressed by existing technologies – Complicated requirements: “run program X at site Y

subject to community policy P, providing access to data at Z according to policy Q”

– High performance: unique demands of advanced & high-performance systems


Elements of the ProblemElements of the Problem

Resource sharing– Computers, storage, sensors, networks, …

– Sharing always conditional: issues of trust, policy, negotiation, payment, …

Coordinated problem solving– Beyond client-server: distributed data analysis,

computation, collaboration, … Dynamic, multi-institutional virtual orgs

– Community overlays on classic org structures

– Large or small, static or dynamic


Building the Grid Building the Grid (according to Ian Foster)(according to Ian Foster)

Open source software– Globus Toolkit® , UK OGSA DAI, Condor, …

Open standards– OGSA, other GGF, IETF, W3C standards, …

Open communities– Global Grid Forum, Globus International, collaborative

projects, … Open infrastructure

– UK eScience, NSF Cyberinfrastructure, StarLight, AP-Grid, …


This talkThis talk



Why Grids?Why Grids? A biochemist exploits 10,000 computers to screen

100,000 compounds in an hour 1,000 physicists worldwide pool resources for

petaop analyses of petabytes of data Civil engineers collaborate to design, execute, &

analyze shake table experiments Climate scientists visualize, annotate, & analyze

terabyte simulation datasets An emergency response team couples real time

data, weather model, population data


Why Grids? (contd)Why Grids? (contd) A multidisciplinary analysis in aerospace couples

code and data in four companies A home user invokes architectural design functions

at an application service provider An application service provider purchases cycles

from compute cycle providers Scientists working for a multinational soap company

design a new product A community group pools members’ PCs to analyze

alternative designs for a local road

July 29, 2003 Grid Computing and Globus, Jennifer M. Schopf 13Image courtesy Harvey Newman, Caltech

Data Grids forData Grids forHigh Energy PhysicsHigh Energy Physics

Tier2 Centre ~1 TIPS

Online System

Offline Processor Farm

~20 TIPS

CERN Computer Centre

FermiLab ~4 TIPSFrance Regional Centre

Italy Regional Centre

Germany Regional Centre

InstituteInstituteInstituteInstitute ~0.25TIPS

Physicist workstations

~100 MBytes/sec

~100 MBytes/sec

~622 Mbits/sec

~1 MBytes/sec

There is a “bunch crossing” every 25 nsecs.

There are 100 “triggers” per second

Each triggered event is ~1 MByte in size

Physicists work on analysis “channels”.

Each institute will have ~10 physicists working on one or more channels; data for these channels should be cached by the institute server

Physics data cache

~PBytes/sec

~622 Mbits/sec or Air Freight (deprecated)




Caltech ~1 TIPS

~622 Mbits/sec

Tier 0Tier 0

Tier 1Tier 1

Tier 2Tier 2

Tier 4Tier 4

1 TIPS is approximately 25,000

SpecInt95 equivalents


Network for EarthquakeNetwork for EarthquakeEngineering Simulation Engineering Simulation

NEESgrid: national infrastructure to couple earthquake engineers with experimental facilities, databases, computers, & each other

On-demand access to experiments, data streams, computing, archives, collaboration

NEESgrid: Argonne, Michigan, NCSA, UIUC, USC


U.S. TeraGridU.S. TeraGrid

NCSA, SDSC, Argonne, Caltech Unprecedented capability

– 13.6 trillion flop/s

– 600 terabytes of data

– 40 gigabits per second

– Accessible to thousandsof scientists working onadvanced research

www.teragrid.org


This talkThis talk



The Globus ProjectThe Globus Project™™

A group of people with a common mission:“Make Grid computing an everyday reality”

Housed at Argonne National Laboratory, Univ. of Chicago, and USC Information Sciences Institute– Led by Ian Foster (ANL, U-C), Carl Kesselman (ISI)

– Includes researchers, software developers, software architects & designers, systems engineers, etc.

– Collaborations (or at least acquaintances) with most Grid activities in the world


Globus Project ActivitiesGlobus Project Activities

All activities contribute to our common mission– Research

– Software Development (prototypes, reference implementations)

– Application consulting

– Infrastructure consulting


The Globus Project cont.The Globus Project cont.

Close collaboration with real Grid projects in both science and industry

The Globus Toolkit®: Open source software base for building Grid infrastructure and applications

Development and promotion of standard Grid protocols and services to enable interoperability and shared infrastructure

Development and promotion of standard Grid software APIs to enable portability and code sharing

Global Grid Forum: We co-founded GGF to foster Grid standardization and community


Globus Project MethodologyGlobus Project Methodology

Identify theoretical applications or user communities. Establish collaborations with target users Identify key requirements of target users Identify common problems & requirements across

many target users Develop architecture and designs for proposed

technological solutions to common problems Implement usable versions of solutions Work with target users to integrate proposed solutions

and evaluate results Propose standards to relevant communities Iterate…


Globus Toolkit (GT)Globus Toolkit (GT)

A software system addressing key technical problems in the development of Grid-enabled tools, services, and applications– Offer a modular set of orthogonal services

– Middleware for building solutions, not turn-key

– Enable incremental development of Grid-enabled tools and applications

– Implement and inform Grid standards

– Available under liberal open source license

– Large community of developers & users

– Multiple commercial support providers


This talkThis talk

What is Grid Computing? Who’s using Grids? What is Globus? What does Globus do?

– Security

– Resource Management

– Information Services

– File Transfer

– OGSA/OGSI Some other resources


Some defintions:Some defintions:

API Protocol


APIAPIApplication Programming InterfaceApplication Programming Interface

A specification for a set of routines to facilitate application development– Refers to definition, not implementation

Often language-specific (or IDL)– Routine name, number, order and type of

arguments; mapping to language constructs

– Behavior or function of routine Examples of APIs

– GSS-API (security), MPI (message passing)


Network ProtocolNetwork Protocol

A formal description of message formats and a set of rules for message exchange– Rules may define sequence of message

exchanges– Protocol may define state-change in

endpoint, e.g., file system state change Good protocols designed to do one thing

– Protocols can be layered Examples of protocols

– IP, TCP, TLS (was SSL), HTTP, Kerberos


A Protocol can have Multiple APIsA Protocol can have Multiple APIs

TCP/IP APIs include BSD sockets, Winsock, System V streams, …

The protocol provides interoperability: programs using different APIs can exchange information

I don’t need to know remote user’s API

TCP/IP Protocol: Reliable byte streams

WinSock API Berkeley Sockets API

Application Application


An API can have Multiple ProtocolsAn API can have Multiple Protocols

An API provides portability: any correct program compiles & runs on a platform

Does not provide interoperability: all processes must link against same SDK–E.g., MPICH and LAM versions of MPI

ApplicationApplication

MPI API MPI API

LAM SDK

LAM protocol

MPICH-P4 SDK

MPICH-P4 protocol

TCP/IP TCP/IPDifferent message formats, exchange

sequences, etc.


Initial Focus On APIsInitial Focus On APIsand Custom Protocolsand Custom Protocols

Primary concern was allowing Grid applications to be built quickly, in order to demonstrate feasibility

Good development APIs and SDKs mattered most

Protocols were a means to an end– We borrowed and extended standard

protocols to make life easier (e.g. LDAP)

– We defined custom protocols (e.g. GRAM)


But Focus Shifted To ProtocolsBut Focus Shifted To Protocols

As demand grew, customers worried about:– compatibility between versions (i.e. Stop

changing the protocols!)– independent implementations of some

components (i.e. What are the protocols?) Ubiquitous adoption demands open, standard

protocols – Internet and Web as guides– Enables innovation/competition on end points– Avoid product/vendor lock-in


GT2GT2Key ProtocolsKey Protocols

The Globus Toolkit v2 (GT2)centers around four key protocols–Security: Grid Security Infrastructure (GSI)

–Resource Management: Grid Resource Allocation Management (GRAM)

– Information Services: Grid Resource Information Protocol (GRIP)

–Data Transfer: Grid File Transfer Protocol (GridFTP)


Why Grid Security is HardWhy Grid Security is Hard Resources being used may be valuable & the problems

being solved sensitive Resources are often located in distinct administrative

domains– Each resource has own policies & procedures

Set of resources used by a single computation may be large, dynamic, and unpredictable– Not just client/server, requires delegation

It must be broadly available & applicable– Standard, well-tested, well-understood protocols;

integrated with wide variety of tools


Grid Security Infrastructure (GSI)Grid Security Infrastructure (GSI) Extensions to standard protocols & APIs

– Standards: SSL/TLS, X.509 & CA, GSS-API– Extensions for single sign-on and delegation

Globus Toolkit reference implementation of GSI– SSLeay/OpenSSL + GSS-API + SSO/delegation– Tools and services to interface to local security

> Simple ACLs; SSLK5/PKINIT for access to K5, AFS; …

– Tools for credential management> Login, logout, etc.> Smartcards> MyProxy: Web portal login and delegation> K5cert: Automatic X.509 certificate creation


X.509 Proxy CertificateX.509 Proxy Certificate

Defines how a short term, restricted credential can be created from a normal, long-term X.509 credential– A “proxy certificate” is a special type of

X.509 certificate that is signed by the normal end entity cert, or by another proxy

– Supports single sign-on & delegation through “impersonation”

– Currently an IETF draft


The Resource Management The Resource Management ChallengeChallenge

Enabling secure, controlled remote access to heterogeneous computational resources and management of remote computation– Authentication and authorization– Resource discovery & characterization– Reservation and allocation– Computation monitoring and control

Addressed by a set of protocols & services– GRAM protocol as a basic building block– Resource brokering & co-allocation services– GSI for security, MDS for discovery


Resource ManagementResource Management

The Grid Resource Allocation Management (GRAM) protocol and client API allows programs to be started on remote resources, despite local heterogeneity

Resource Specification Language (RSL) is used to communicate requirements

A layered architecture allows application-specific resource brokers and co-allocators to be defined in terms of GRAM services– Integrated with Condor, PBS, MPICH-G2, …


GRAM ProtocolGRAM Protocol

GRAM-1: Simple HTTP-based RPC– Job request

> Returns a “job contact”: Opaque string that can be passed between clients, for access to job

– Job cancel, status, signal

– Event notification (callbacks) for state changes> Pending, active, done, failed, suspended

GRAM-1.5 (U Wisconsin contribution)– Add reliability improvements

> Once-and-only-once submission

> Recoverable job manager service

> Reliable termination detection


GT2 ImplementationGT2 Implementation

Gatekeeper– Single point of entry– Authenticates user, maps to local security

environment, runs service– In essence, a “secure inetd”

Job manager– A gatekeeper service– Layers on top of local resource management

system (e.g., PBS, LSF, etc.)– Handles remote interaction with the job


MDS: MDS: Monitoring and Discovery ServiceMonitoring and Discovery Service

Globus Information Service Requirements and characteristics

– Uniform, flexible access to information

– Scalable, efficient access to dynamic data

– Access to multiple information sources

– Decentralized maintenance

– Secure information provision


MDS ArchitectureMDS Architecture Resources run a standard information service (GRIS) that speaks

LDAP and provides information about the resource GIIS provides a “caching” service

– Resources register with GIIS– GIIS pulls information when requested by a client (when out of date)

GIIS provides the collective-level indexing/searching function

GIIS

Cache contains info from A and B

Resource A

GRIS

GRIS register with GIIS

GIIS requests infofrom GRIS services

Client 1Client 2

Resource B

GRIS

Client 1 requests infodirectly from resources. Client 2 uses GIIS for searching

collective information.


ProtocolsProtocols

MDS protocols based on LDAP Dynamic Registration via Reg. Protocol (GRRP)

– soft-state protocol Resource Inquiry via Info. Protocol (GRIP)

– Co-located with resource on network Resource Discovery (via GRIP or other)

– Using GRIP allows resource/directory hierarchy Also well defined interfaces to add new sensor data


A Model Architecture for Data GridsA Model Architecture for Data Grids

Metadata Catalog

Replica Catalog

Tape Library

Disk Cache

Attribute Specification

Logical Collection and Logical File Name

Disk Array Disk Cache

Application

Replica Selection

Multiple Locations

NWS

SelectedReplica

GridFTP Control ChannelPerformanceInformation &Predictions

Replica Location 1 Replica Location 2 Replica Location 3

MDS

GridFTPDataChannel


Data Management - GridFTPData Management - GridFTP

Secure: uses GSI Fast: parallelism (multiple TCP streams),

striping (multiple hosts), TCP buffer control, data channel caching

Robust: Enhanced restart in the face of failure, plug-ins

Other: 3rd Party Transfer, Server Side Processing, Integrated Instrumentation


Data Management StandardsData Management Standards

GridFTP is based on several existing standards:– RFC 959: File Transfer Protocol

– RFC 2228: FTP Security Extensions

– RFC 2389: Feature Negotiation (FEAT,OPTS)

– Draft: structured file listing, MODE S restart New drafts

– GridFTP: Protocol Extensions to FTP for the Grid> Draft before the Grid Forum Working Group


From Standard Protocols to Grid From Standard Protocols to Grid ServicesServices

Heterogeneous protocol base was hurting us Increasing number of virtual services that

needed to be managed Web services (WSDL, SOAP) appeared


Incr

ease

d fu

nctio

nalit

y,st

anda

rdiz

atio

n

Customsolutions

1990 1995 2000 2005

Open GridServices Arch

Real standardsMultiple implementations

Web services, etc.

Managed sharedvirtual systems

Academic and industry R&D

Globus Toolkit

Defacto standardSingle implementation

Internetstandards

The Evolution ofThe Evolution ofGrid Technologies and StandardsGrid Technologies and Standards

2010


Heterogenous Protocol BaseHeterogenous Protocol Base

Our core protocols (GRAM, LDAP, GridFTP) had overlapping but different functionality– E.g. Each allows monitoring, but in different ways

and with different functionality But we increasingly wanted to integrate across

protocols– E.g. Generic monitoring services (archival and

replay, correlation, etc.) that could work with all of these core protocols

A common protocol base sure would be convenient…


Managing Virtual ServicesManaging Virtual Services Trying to manage total system properties

– E.g. Dependability, end-to-end QoS “Resource” tends to connote a tangible entity to be

consumed: cpu, storage, bandwidth, … But many interesting services may be decoupled from

any particular resource– E.g. Finite element analysis service– A service consumes resources, but how that happens is

irrelevant to the client “Service” forms a better base abstraction

– Can apply to physical or virtual


ServiceService

Implementation of a protocol that defines a set of capabilities–Protocol defines interaction with service

–All services require protocols

–Not all protocols are used to provide services (e.g. IP, TLS)

Examples: FTP and Web serversWeb Server

IP Protocol

TCP Protocol

TLS Protocol

HTTP Protocol

FTP Server

IP Protocol

TCP Protocol

FTP Protocol

Telnet Protocol


Service DefinitionService Definition

Service definition = abstract interface + semantics– Interface implies protocol, through standard

binding definitions

– Can be mapped to language-specific APIs> Can be automated for multiple languages

This is obviously not new– E.g. CORBA: IDL + IIOP binding


Transient Service InstancesTransient Service Instances “Web services” address discovery & invocation of

persistent services– Interface to persistent state of entire enterprise

In Grids, must also support transient service instances, created/destroyed dynamically– Interfaces to the states of distributed activities– E.g. workflow, video conf., dist. data analysis

Significant implications for how services are managed, named, discovered, and used– In fact, much of Grid is concerned with the management of

service instances


Grid Evolution:Grid Evolution:Open Grid Services ArchitectureOpen Grid Services Architecture

Refactor Globus protocol suite to enable common base and expose key capabilities

Service orientation to virtualize resources and unify resources/services/information

Embrace key Web services technologies – WSDL: Language for defining abstract service interfaces– SOAP (and friends): Binding from WSDL to bytes on the

wire– Address discovery & invocation of persistent services

Grids also need transient service instances Result: standard interfaces & behaviors for

distributed system management: the Grid service


OGSA StructureOGSA Structure

A standard substrate: the Grid service– OGSI = Open Grid Service Infrastructure

– Standard interfaces and behaviors that address key distributed system issues

– Much borrowed from GT abstractions … supports standard service specifications

– Resource mgt, dbms, workflow, security, …

– Target of current & planned GGF efforts … and arbitrary application-specific services based on

these & other definitions


OGSI Grid Service SpecificationOGSI Grid Service Specification

Defines WSDL conventions and GSDL extensions– For describing and structuring services

– Working with W3C WSDL working group to drive GSDL extensions into WSDL

Defines fundamental interfaces (using WSDL) and behaviors that define a Grid Service– A unifying framework for interoperability &

establishment of total system properties


Standard Interfaces & Behaviors:Standard Interfaces & Behaviors:Four Interrelated ConceptsFour Interrelated Concepts

Naming and bindings– Every service instance has a unique name, from which

can discover supported bindings Lifecycle

– Service instances created by factories– Destroyed explicitly or via soft state

Information model– Service data associated with Grid service instances,

operations for accessing this info– Basis for service introspection, monitoring, discovery

Notification– Interfaces for registering existence, and delivering

notifications of changes to service data


Example:Example:Reliable File Transfer ServiceReliable File Transfer Service

Performance

Policy

Faults

servicedataelements

Pending

GridService

Notf’nSource

FileTransfer

Policy

InternalState

Query &/orsubscribe

to service data

interfaces

Client Client Client

FaultMonitor

Perf.Monitor

Request and manage file transfer operations

Data transfer operations


GT2 Evolution To GT3GT2 Evolution To GT3

ALL of GT2 functionality is in GT3 What happened to the GT2 key protocols?

– Security: Adapting X.509 proxy certs to integrate with emerging WS standards

– GRIP/LDAP: Abstractions integrated into OGSI as serviceData

– GRAM: ManagedJobFactory and related service definitions

– GridFTP: Unchanged in 3.0, but will evolve into OGSI-compliant service in 2004

Also rendering collective services in terms of OGSI: RFT, RLS, CAS, etc.


This talkThis talk


– NMI GRIDS center– Grid Technology Repository (GTR)– Global Grid Forum (GGF)– General support info


GRIDS Center (NMI)GRIDS Center (NMI)

GRIDS Center– GRIDS = Grid Research Integration Development and

Support– Partnership of leading teams in Grid computing– Funded by NSF Middleware Initiative (NMI)

Goal: Design, Develop, Deploy and Support

– Define an integrated, modular architecture that addresses current & projected middleware requirements for the S&E communities

– Create robust, tested, packaged, documented, and well-supported middleware solutions that are extensible within and beyond S&E


GRIDS CenterGRIDS CenterSoftware SuiteSoftware Suite

Globus Toolkit Condor-G

– Enhanced version of the core Condor software optimized to work with GT for managing Grid jobs.

Network Weather Service (NWS)– Monitors and dynamically forecasts performance of network

and computational resources. Grid Packaging Tools (GPT)

– XML-based packaging data format defines complex dependencies between components.

GSI-OpenSSH– Modified version adds support for Grid Security Infrastructure

(GSI) authentication and single sign-on capability


GRIDS CenterGRIDS CenterSoftware Suite (cont.)Software Suite (cont.)

MyProxy– Repository lets users retrieve a proxy credential on

demand, without managing private key and certificate files across sites and applications.

MPICH-G2– Grid-enabled implementation of the Message Passing Index

(MPI) standard, based on the popular MPICH library. GridConfig

– Manages the configuration of GRIDS components, letting users regenerate configuration files in native formats and ensure consistency.

KX.509 and KCA– A tool from EDIT that bridges Kerberos and PKI

infrastructure.


GRIDS platformsGRIDS platforms

NMI-R3.1 maintenance release supports:– Red Hat Linux 7.2, 7.3, 8.0 and 9.0 on IA32

– Red Hat Linux 7.2 on IA64

– SuSE Linux Enterprise Server 8 on IA64

– Solaris 8.0 on SPARC Distributed as binaries

– Source distribution is available, but is not officially supported


Grid Technology Repository (GTR)Grid Technology Repository (GTR)

Repository of code, documents, etc. – related to Grid computing, and free to the community– Additional information providers– Service data browser for GT3– Documentation on deployment strategies– And more!

International, community-driven effort, with contributions welcome from academia, industry and individuals without institutional affiliation

Contributions are available on a "use at your own risk" basis

http://gtr.globus.org


Global Grid Forum (GGF)Global Grid Forum (GGF)

An Open Process for Development of Standards– Grid “Recommendations” process modeled after Internet

Standards Process (IETF)– Persistent, Reviewed Document Series (similar to RFC)

A Forum for Information Exchange– Experiences, patterns, structures– Useful even if every application & Grid were completely

separate and not interoperable…but ideally will result in interoperability!

A Regular Gathering to Encourage Shared Effort– In code development: libraries, tools…– Via resource sharing: shared Grids– In infrastructure: consensus standards

http://www.ggf.org


General Help and SupportGeneral Help and Support

Globus-discuss list– [email protected]

– http://globus.org/about/contacts.html Bugzilla

– Bugzilla.globus.org


Upcoming Globus PlansUpcoming Globus Plans

GT 3.O RELEASED June 2002!– Address transition & operations issues

GT 3.2 release end of 2003, early 2004– New GridFTP server, Community access

service, better index service, plus bug fixes GlobusWorld 2004 in San Francisco

– January 20-23

– http://www.globusworld.org


Recap: Why You Should CareRecap: Why You Should Care

Grid Computing means sharing resources for coordinated problem solving

Many applications are using this approach Globus is the defacto standard

– Security, resource management, information services, file transfer and more

– OGSA/OGSI


Jennifer Schopf– [email protected]

The Globus Project™– www.globus.org

Technical articles– www.globus.org/research/

papers.html Open Grid Services Arch.

– www.globus.org/ogsa Global Grid Forum

– www.ggf.org NMI GRIDS center

– www.grids-center.org

For More InformationFor More Information

2nd Edition to appear November 2003

grid computing and the globus toolkit jennifer m. schopf argonne national lab

Documents

grid computing

internet computing

grid foster

ap grid

globus toolkit jennifer

globus international

globus gt2gt3

s heterogeneous computing