1siac 2000, wright state university, august 21, 2000 1 wisdom consists of knowing when to avoid...
TRANSCRIPT
1SIAC 2000, Wright State University, August 21, 20001
Wisdom consists of knowing when to avoid perfection.
- Horowitz
Tuesday, February 24, 2009
The Globus Project
Infrastructure for Computational Grids
The Globus Project Teamhttp://www.globus.org/
3SIAC 2000, Wright State University, August 21, 2000
Session Goals
• Provide an introduction to… computational grids the capabilities of the Globus Toolkit pragmatic issues with grids & Globus
• Enable attendees to… start building grids using the Globus Toolkit start building & using grid applications
4SIAC 2000, Wright State University, August 21, 2000
Overview• Introduction to computational grids
• Introduction to the Globus Toolkit Portability Security Information services Resource management Data management Communication
• Case studies
• Other Globus services, and future directions
5SIAC 2000, Wright State University, August 21, 2000
What is a computational grid?
• A pool of computational resources that can be “plugged into” via standard interfaces.
• Processors• Data storage devices• Instruments
• As the power grid is to electrical power, and the telephone grid is to voice communication, so will the computational grid be for computation.
22SIAC 2000, Wright State University, August 21, 2000
Globus Project Participants
• Globus Project is a large community effort Globus Toolkit core development
• Argonne, USC/ISI, NCSA, SDSC
Globus Toolkit contributors• NASA, DOE ASCI DRM (SNL, LBNL, LLNL), Raytheon, and
numerous others
Collaborators• University, lab, industrial, and international partners
spanning many scientific and engineering disciplines
• Active in Grid Forum http://www.gridforum.org
23SIAC 2000, Wright State University, August 21, 2000
Globus Approach
• A toolkit and collection of services addressing key technical problems Modular “bag of services” model Not a vertically integrated solution General infrastructure tools (aka middleware)
that can be applied to many application domains
• Inter-domain issues, rather than clustering Integration of intra-domain solutions
• Distinguish between local and global services
24SIAC 2000, Wright State University, August 21, 2000
Globus Hourglass
• Focus on architecture issues Propose set of core services
as basic infrastructure Use to construct high-level,
domain-specific solutions
• Design principles Keep participation cost low Enable local control Support for adaptation “IP hourglass” model
Diverse global services
Core Globusservices
Local OS
A p p l i c a t i o n s
25SIAC 2000, Wright State University, August 21, 2000
Technical Focus & Approach
• Enable incremental development of grid-enabled tools and applications Model neutral: Support many programming models,
languages, tools, and applications Evolve in response to user requirements
• Deploy toolkit on international-scale production grids and testbeds Large-scale application development & testing
• Information-rich environment Basis for configuration and adaptation
26SIAC 2000, Wright State University, August 21, 2000
Layered Architecture
Applications
Grid ServicesGRAM
GSI HBM
Nexus
globus_io
Grid Fabric
LSF
Condor MPI
NQEPBS
TCP
NTLinux
UDP
Application Toolkits
DUROC globusrunMPICH-G Nimrod/GCondor-G HPC++
GlobusView Web Portals
GASS
Solaris DiffServ
GSI-FTPMDS
27SIAC 2000, Wright State University, August 21, 2000
Globus Toolkit Grid Services
• Security (GSI)
• Information services (MDS)
• Resource management (GRAM)
• Data management (GASS, GSI-FTP, replicas)
• Communication (globus_io, Nexus)
• Fault detection (HBM)
• Portability (globus_dc, globus_thread)
28SIAC 2000, Wright State University, August 21, 2000
Other Globus Project Grid Services
• Coming Soon Data transfer (GSI-FTP) Replica Management
http://www.globus.org/datagrid
• Experimental Prototypes Advanced Reservations & QoS (GARA) Distributed Events & Logging
29SIAC 2000, Wright State University, August 21, 2000
Sample of High-Level Services
• Resource brokers and co-allocators DUROC, HTB, Nimrod/G, Condor-G, ASCI DRM
• Communication & I/O libraries MPICH-G, PAWS, RIO (MPI-IO), PPFS, MOL
• Parallel languages HPC++, CC++
• Collaborative environments CAVERNsoft, ManyWorlds
• Others MetaNEOS, NetSolve, LSA, AutoPilot, WebFlow
30SIAC 2000, Wright State University, August 21, 2000
Condor-G: Condor for the Grid• Condor is a high-throughput scheduler
• Condor-G uses Globus Toolkit libraries for: Security (GSI) Managing remote jobs on Grid (GRAM) File staging & remote I/O (GSI-FTP)
• Grid job management interface & scheduling Robust replacement for Globus Toolkit programs
• Globus Toolkit focus is on libraries and services, not end user vertical solutions
Supports single or high-throughput apps on Grid• Personal job manager which can exploit Grid resources
31SIAC 2000, Wright State University, August 21, 2000
Production Grids & Testbeds
• Production deployments underway at: NSF PACIs (National Technology Grid) NASA Information Power Grid DOE ASCI European Grid
• Research testbeds EMERGE: Advance reservation & QoS GUSTO: Globus Ubiquitous Supercomputing Testbed
Organization Particle Physics Data Grid Earth Systems Grid
32SIAC 2000, Wright State University, August 21, 2000
Production Grids & Testbeds
NASA’s Information Power Grid The Alliance National Technology Grid
GUSTO Testbed
33SIAC 2000, Wright State University, August 21, 2000
Application Experiments
• Computed microtomography (ANL, ISI) Real-time, collaborative analysis of data from X-
Ray source (and electron microscope)
• Hydrology (ISI, UMD, UT; also NCSA, Wisc.) Interactive modeling and data analysis
• Collaborative engineering (“tele-immersion”) CAVERNsoft @ EVL
• OVERFLOW (NASA) Large CFD simulations for aerospace vehicles
34SIAC 2000, Wright State University, August 21, 2000
Application Experiments
• Distributed interactive simulation (CIT, ISI) Record-setting SF-Express simulation
• Cactus Astrophysics simulation, viz, and steering Including trans-Atlantic experiments
• Particle Physics Data Grid High Energy Physics distributed data analysis
• Earth Systems Grid Climate modeling data management
35SIAC 2000, Wright State University, August 21, 2000
Where Are We? (August 2000)
• Research is focused on data management, resource management, and web portals.
• Globus Toolkit v4 has been released. Runs on most versions of Unix, Win32 clients.
• Production deployment is underway. NSF PACIs, NASA IPG, DOE ASCI DRM
• Many research applications and tools are using these testbeds.
• We’re always looking for interesting applications.
36SIAC 2000, Wright State University, August 21, 2000
For More Information on Globushttp://www.globus.org/
• Papers on most components
• Tutorials User, Developer, Administrator
• Manuals Quick Start Guide, System Administration Guide
• Mailing lists [email protected], [email protected]
• Software & API documentation
• Application descriptions
• Attend Supercomputing 2000 (Nov. 2000)
37SIAC 2000, Wright State University, August 21, 2000
The Grid:Blueprint for a New Computing Infrastructure
I. Foster, C. Kesselman (Eds), Morgan Kaufmann, 1999
• Available July 1998;
ISBN 1-55860-475-8 • 22 chapters by expert
authors including Andrew Chien, Jack Dongarra, Tom DeFanti, Andrew Grimshaw, Roch Guerin, Ken Kennedy, Paul Messina, Cliff Neuman, Jon Postel, Larry Smarr, Rick Stevens, and many others
http://www.mkp.com/grids
“A source book for the historyof the future” -- Vint Cerf
38SIAC 2000, Wright State University, August 21, 2000
Session Approach
• Five sections, each illustrating a basic Globus service
• Laboratory material is available to allow practice with the use of each technique See http://www.globus.org/tutorial/
39SIAC 2000, Wright State University, August 21, 2000
Desktop Supercomputing• Seamlessly, from the desktop
Sign-on once Locate available computers Start computation on an appropriate
system Monitor progress Get output files Manipulate locally
• E.g. ECCE’, Cactus, Hotpage, Chemical Eng. Workbench, WebFlow, LSA
40SIAC 2000, Wright State University, August 21, 2000
WebFlow Grid Interface
• Dataflow computing interface to grid computing Fox, Haupt: Syracuse
• Globus services for Authentication Process creation and
management
• Applications include nanomaterials
41SIAC 2000, Wright State University, August 21, 2000
Application Challenges
• Security How do we authenticate ourselves at the remote
site?
• Resource specification How do we locate and request a resource?
• Staging of code and data How do we stage a user’s executables and data to
the remote resource?
• Computation How do we start & manage computation?
42SIAC 2000, Wright State University, August 21, 2000
Grid Services
• Single sign-on for all resources No need for user to keep track of accounts and
passwords at multiple sites No plaintext passwords
• Uniform interface to various local scheduling mechanisms PBS, Condor, LSF, NQE, LoadLeveler, fork, etc. No need to learn and remember obscure
command sequences at different sites
• Support for file staging, remote I/O, etc.
43SIAC 2000, Wright State University, August 21, 2000
Grid Authentication Model
• Authentication is done on a “user” basis Single authentication step allows access to all grid
resources
• No communication of plaintext passwords
• Most sites will use conventional account mechanisms You must have an account on a resource to use that
resource
• Sites may use “generic” Grid accounts Not common, but Globus can deal with it
44SIAC 2000, Wright State University, August 21, 2000
Grid Security Infrastructure (GSI)
• Based on public key technology Standard X.509 certificate, same as
certificates used for the Web
• Each user has: a Grid user id (called a Subject Name) a private key (like a password) a certificate signed by a Certificate
Authority (CA)
• A “gridmap” file at each site specifiesgrid-id to local-id mapping
45SIAC 2000, Wright State University, August 21, 2000
Certificate Based Authentication• User has a certificate, signed by a trusted “certificate
authority” (CA) Certificate contains users name and public key Globus project operates a CA
• User’s private key is used to encode a challenge string
• Public key is used to decode the challenge If you can decode it, you know the user
• Treat your private key carefully!! Private key is stored in encrypted form
46SIAC 2000, Wright State University, August 21, 2000
User Proxies
• Minimize exposure of user’s private key• A temporary credential for use by our
computations We call this a user proxy certificate Allows process to act on behalf of user User-signed user proxy certificate stored in
local file
• Proxy’s private key is not encrypted Rely on file system security, proxy certificate
file must be readable only by the owner
47SIAC 2000, Wright State University, August 21, 2000
Delegation
• Remote creation of a user proxy
• Allows remote process to act on behalf of the user
• Avoids sending passwords or private keys across the network
48SIAC 2000, Wright State University, August 21, 2000
Single sign-onvia “grid-id”
User
User Proxy
GlobusGlobusCredentialCredential
Site 1
Kerberos
GRAM Process
Process
ProcessGSI
TicketTicket
Site 2
Public Key
GRAM
GSI
CertificateCertificate
Process
Process
Process
Authenticatedinterprocess
communication
CREDENTIAL
GSSAPI:multiplelow-level
mechanisms
Mutualuser-resourceauthentication
Mappingto local ids
Assignment of credentials to“user proxies”
49SIAC 2000, Wright State University, August 21, 2000
Globus Authentication Setup• Before you can run Globus applications:
Install Globus Obtain a Grid certificate and key Set up your environment so Globus knows where to
find certificates and keys Contact sites to set up local accounts and globusmap
entries Create proxy certificate for each application run
• Documentation Globus Quick Start Guide (on website)
55SIAC 2000, Wright State University, August 21, 2000
Simple job submission
• globus-job-run provides a simple RSH compatible interface% grid-proxy-init Enter PEM pass phrase: *****% globus-job-run host program [args]
• Job submission will be covered in more detail in Part 5
56SIAC 2000, Wright State University, August 21, 2000
Exercise: Sign-On & Remote Process Creation
• Use grid-proxy-init to create a proxy certificate:
% grid-proxy-initEnter PEM pass phrase:......................................+++++.....+++++
• Use grid-proxy-info to query proxy:% grid-proxy-info -subject
• Use globus-job-run to start remote programs:
% globus-job-run jupiter.isi.edu /usr/bin/ls -l /tmp
57SIAC 2000, Wright State University, August 21, 2000
Globus Components Being Used
• GSI: Grid Security Infrastructure Authenticate to remote system
• GRAM: Globus Resource Allocation Manager Create process on remote resource, deal with
local resource managers
• GASS: Global Access to Secondary Storage Redirect standard output
(More on GRAM and GASS later!)
58SIAC 2000, Wright State University, August 21, 2000
Job Submission Interfaces
• Globus Toolkit includes several command line programs for job submission globus-job-run: Interactive jobs globus-job-submit: Batch/offline jobs globusrun: Flexible scripting infrastructure
• Others are building better interfaces General purpose
• Condor-G, PBS, GRD, Hotpage, etc
Application specific• ECCE’, Cactus, Web portals
59SIAC 2000, Wright State University, August 21, 2000
globus-job-run
• For running of interactive jobs• Additional functionality beyond rsh
Ex: Run 2 process job w/ executable stagingglobus-job-run -: host –np 2 –s myprog arg1 arg2
Ex: Run 5 processes across 2 hostsglobus-job-run \
-: host1 –np 2 –s myprog.linux arg1 \-: host2 –np 3 –s myprog.aix arg2
For list of arguments run:globus-job-run -help
60SIAC 2000, Wright State University, August 21, 2000
globus-job-submit
• For running of batch/offline jobs globus-job-submit Submit job
• Same interface as globus-job-run• Returns immediately
globus-job-status Check job status globus-job-cancel Cancel job globus-job-get-output Get job stdout/err globus-job-clean Cleanup after job
61SIAC 2000, Wright State University, August 21, 2000
globusrun
• Flexible job submission for scripting Uses an RSL string to specify job request Contains an embedded globus-gass-server
• Defines GASS URL prefix in RSL substitution variable:
(stdout=$(GLOBUSRUN_GASS_URL)/stdout)
Supports both interactive and offline jobs
• Complex to use Must write RSL by hand Must understand its esoteric features Generally you should use globus-job-* commands
instead
62SIAC 2000, Wright State University, August 21, 2000
Summary
• Grid Security Infrastructure (GSI) provides single sign-on capability
• globus-job-run can be used to create a remote process Difference between schedulers managed by
Globus Strong authentication provided
• Remote process creation can be added to applications by using Globus services
• Rest [Self Study]
68SIAC 2000, Wright State University, August 21, 2000
MDS Features• White Pages
Look up the IP number, amount of memory, etc., associated with a particular machine
• Yellow Pages Search for computers of a particular class or with a
particular property • Information is dynamic!
In a distributed system, things change without warning. Information often has an expiration date or other
measures of uncertainty.
69SIAC 2000, Wright State University, August 21, 2000
MDS Approach
• Based on LDAP Lightweight Directory Access
Protocol v3 (LDAPv3) Standard data model Standard query protocol
• Globus specific schema Host-centric representation
• Globus specific tools GRIS, GIIS Data discovery, publication,…
SNMP
GRIS
NIS
NWS
LDAP
LDAP API
Middleware
…
Application
GIIS…
70SIAC 2000, Wright State University, August 21, 2000
LDAP Details
• Lightweight Directory Access Protocol Stripped down version of X.500 DAP protocol Supports distributed storage/access (referrals) Supports replication Becoming de facto standard
• Defines: Network protocol for accessing directory contents Information model defining form of information Namespace defining how information is
referenced and organized
71SIAC 2000, Wright State University, August 21, 2000
LDAP Directory Structure
• Directory contents Called Object Classes and Entries What information is stored in directory Group related information into entries
• Directory organization Called Directory Information Tree (DIT) Objects are organized into tree structure Position of object in tree uniquely names
entry within the server
72SIAC 2000, Wright State University, August 21, 2000
Sample Object Classes
• Compute Resources Operating System Memory Hierarchy Health and Status
• Network Interfaces IP address Interface types
• Performance Data Schedule Jobs CPU Loads Network Traffic
• Resource Managers Contact strings Scheduled jobs Free nodes
• Software Configuration Version Control Contact information
• Organizations
• People
73SIAC 2000, Wright State University, August 21, 2000
LDAP Directory Information Tree
• Directory entries are organized into a tree. Called Directory Information Tree (DIT) Subtrees can be distributed or replicated.
• Position in tree uniquely names entry within a server.
• Each object in a server is uniquely determined by its distinguished name (DN). List of unique attribute names and values
along path from root of DIT to object, e.g.:<hn=sp2.sdsc.edu, dc=sdsc, dc=edu, o=Grid>
83SIAC 2000, Wright State University, August 21, 2000
MDS Tools
• Java LDAP browser http://www.mcs.anl.gov/~gawor/ldap
• Web-based browsers and displays http://www.globus.org/mds
• CGI-based MDS browser• MDS Object Class Browser
• Various APIs and search tools • Translators from “Globus Object Definition
Language” Commented LDIF - LDAP schema definition Converts to LDIF, HTML
84SIAC 2000, Wright State University, August 21, 2000
85SIAC 2000, Wright State University, August 21, 2000
MDS Access/Update Commands
• LDAP defines a set of standard commandsldapsearch, ldapmodify, ldapdelete, etc.
• We also define MDS-specific commands grid-info-search, grid-info-create, grid-info-update, grid-
info-remove, grid-info-host-search Routines to ensure data consistency and to insert
metadata
• APIs are defined for C, Java, etc. C: OpenLDAP client API
• ldap_search_s(), ldap_modify_s(), …
Java: JNDI
97SIAC 2000, Wright State University, August 21, 2000
Resource Management
• Resource Specification Language (RSL) is used to communicate requirements
• The Globus Resource Allocation Manager (GRAM) API allows programs to be started on remote resources, despite local heterogeneity
• A layered architecture allows application-specific resource brokers and co-allocators to be defined in terms of GRAM services
99SIAC 2000, Wright State University, August 21, 2000
Resource Specification Language
• Common notation for exchange of information between components Syntax similar to MDS/LDAP filters
• RSL provides two types of information: Resource requirements: Machine type,
number of nodes, memory, etc. Job configuration: Directory, executable,
args, environment
• API provided for manipulating RSL
100SIAC 2000, Wright State University, August 21, 2000
RSL Syntax
• Elementary form: parenthesis clauses (attribute op value [ value … ] )
• Operators Supported: <, <=, =, >=, > , !=
• Some supported attributes: executable, arguments, environment, stdin, stdout,
stderr, resourceManagerContact,resourceManagerName
• Unknown attributes are passed through May be handled by subsequent tools
107SIAC 2000, Wright State University, August 21, 2000
Job Submission Interfaces
• Globus Toolkit includes several command line programs for job submission globus-job-run: Interactive jobs globus-job-submit: Batch/offline jobs globusrun: Flexible scripting infrastructure
• Others are building better interfaces General purpose
• Condor-G, PBS, GRD, Hotpage, etc
Application specific• ECCE’, Cactus, Web portals