the anatomy of the grid: an integrated view of grid architecture ian foster, steve tuecke argonne...
TRANSCRIPT
![Page 1: The Anatomy of the Grid: An Integrated View of Grid Architecture Ian Foster, Steve Tuecke Argonne National Laboratory The University of Chicago Carl Kesselman](https://reader034.vdocument.in/reader034/viewer/2022051315/56649ed25503460f94be267b/html5/thumbnails/1.jpg)
The Anatomy of the Grid: An Integrated View of Grid
Architecture
Ian Foster, Steve TueckeArgonne National Laboratory
The University of Chicago
Carl KesselmanUSC/Information Sciences Institute
![Page 2: The Anatomy of the Grid: An Integrated View of Grid Architecture Ian Foster, Steve Tuecke Argonne National Laboratory The University of Chicago Carl Kesselman](https://reader034.vdocument.in/reader034/viewer/2022051315/56649ed25503460f94be267b/html5/thumbnails/2.jpg)
Outline and Goals
Define what is (and is not) a Grid in terms of a layered protocol architecture
Understand the role of protocols, services, APIs and tools in the deployment and use of Grids
Illustrate how a layered protocol architecture works with specific examples
Propose a view of where the Grid Forum can play a particularly important role
![Page 3: The Anatomy of the Grid: An Integrated View of Grid Architecture Ian Foster, Steve Tuecke Argonne National Laboratory The University of Chicago Carl Kesselman](https://reader034.vdocument.in/reader034/viewer/2022051315/56649ed25503460f94be267b/html5/thumbnails/3.jpg)
What is a Grid?
Grid language has been driven by genesis from metacomputing, but…
In practice, the Grid is about resource sharing and coordinated problem solving in dynamic, multi-institutional virtual organizations
Focus on how to enable, maintain and control the sharing of resources to achieve a common goal
![Page 4: The Anatomy of the Grid: An Integrated View of Grid Architecture Ian Foster, Steve Tuecke Argonne National Laboratory The University of Chicago Carl Kesselman](https://reader034.vdocument.in/reader034/viewer/2022051315/56649ed25503460f94be267b/html5/thumbnails/4.jpg)
Some Useful Definitions
Network Protocol A formal description of message formats and
a set of rules for exchange of messages Rules define sequences of message
exchange, and potentially resulting behavior Protocol may define state-change in endpoint
Network Enabled Services Defines a set of capabilities Protocol defines interaction with service
All services require protocols, although not all protocols are to services
![Page 5: The Anatomy of the Grid: An Integrated View of Grid Architecture Ian Foster, Steve Tuecke Argonne National Laboratory The University of Chicago Carl Kesselman](https://reader034.vdocument.in/reader034/viewer/2022051315/56649ed25503460f94be267b/html5/thumbnails/5.jpg)
More definitions
Resource Entity that is to be shared Provides some capabilities, that can be
accessed via interface (API) or protocol Application Programmer Interface (API) Software Development Kit (SDK)
Package that enables application development, consisting of one or more APIs, and programming tools
![Page 6: The Anatomy of the Grid: An Integrated View of Grid Architecture Ian Foster, Steve Tuecke Argonne National Laboratory The University of Chicago Carl Kesselman](https://reader034.vdocument.in/reader034/viewer/2022051315/56649ed25503460f94be267b/html5/thumbnails/6.jpg)
Protocols Make the Grid
Protocols vs. APIs Protocols enable interoperability APIs enable portability
Sharing is about interoperability, so … Grid architecture should be about protocols
By extension, a major role for the Grid Forum is to define protocols where standards do not exist
![Page 7: The Anatomy of the Grid: An Integrated View of Grid Architecture Ian Foster, Steve Tuecke Argonne National Laboratory The University of Chicago Carl Kesselman](https://reader034.vdocument.in/reader034/viewer/2022051315/56649ed25503460f94be267b/html5/thumbnails/7.jpg)
Grid Services Architecture: Previous Perspective
Grid-enabled archives, networks, computers, display devices, etc.; associated local services
Protocols, authentication, policy, resource management, instrumentation, discovery, etc., etc.
GridFabric
GridServices
ApplnToolkits
Applns
...
… a rich variety of applications ...
Remoteviz
toolkit
Remotecomp.toolkit
Remotedata
toolkit
Remotesensorstoolkit
Async.collab.toolkit
![Page 8: The Anatomy of the Grid: An Integrated View of Grid Architecture Ian Foster, Steve Tuecke Argonne National Laboratory The University of Chicago Carl Kesselman](https://reader034.vdocument.in/reader034/viewer/2022051315/56649ed25503460f94be267b/html5/thumbnails/8.jpg)
Characteristics of Grid Services Architecture
Identifies separation of concerns Isolates Grids from languages and specific
programming environments Makes provisions for generic and
application specific functionality Protocols not explicit in architecture
Fails to make clear distinction between language, service, and networking issues
![Page 9: The Anatomy of the Grid: An Integrated View of Grid Architecture Ian Foster, Steve Tuecke Argonne National Laboratory The University of Chicago Carl Kesselman](https://reader034.vdocument.in/reader034/viewer/2022051315/56649ed25503460f94be267b/html5/thumbnails/9.jpg)
Grid Services Architecture v2.0:Layered Grid Protocol Architecture
Connectivity
Fabric
Resource
Grid
Application
User
![Page 10: The Anatomy of the Grid: An Integrated View of Grid Architecture Ian Foster, Steve Tuecke Argonne National Laboratory The University of Chicago Carl Kesselman](https://reader034.vdocument.in/reader034/viewer/2022051315/56649ed25503460f94be267b/html5/thumbnails/10.jpg)
Important Points
Being Grid-enabled requires speaking appropriate protocols Protocol only requirement, not reachability Protocols can be used to bridge local resources
or “local Grids” Talk about Intergrid (analog to Internet)
Built on Internet protocols Independent of language and implementation
Focus on interaction over network Services exist at each level
![Page 11: The Anatomy of the Grid: An Integrated View of Grid Architecture Ian Foster, Steve Tuecke Argonne National Laboratory The University of Chicago Carl Kesselman](https://reader034.vdocument.in/reader034/viewer/2022051315/56649ed25503460f94be267b/html5/thumbnails/11.jpg)
Grid Services Architecture v2.0:Protocols, Services, and Interfaces
Languages/Frameworks
Fabric Layer
Applications
Local Access APIs and protocols
Grid Service APIs and SDKs
Grid ServicesGrid Service Protocols
Resource APIs and SDKs
Resource ServicesResource Service Protocols
User Service ProtocolsUser Service APIs and SDKs
User Services
Connectivity APIs
Connectivity Protocols
![Page 12: The Anatomy of the Grid: An Integrated View of Grid Architecture Ian Foster, Steve Tuecke Argonne National Laboratory The University of Chicago Carl Kesselman](https://reader034.vdocument.in/reader034/viewer/2022051315/56649ed25503460f94be267b/html5/thumbnails/12.jpg)
The Integrated Grid Architecturein Practice
We now proceed to illustrate the general concepts outlined above by reference to specific examples First, we analyze some extant systems in
terms of this architecture Then, we review example protocols,
services, APIs, SDKs in each layer Apology: We will necessarily need to refer
to some specific protocols and systems
![Page 13: The Anatomy of the Grid: An Integrated View of Grid Architecture Ian Foster, Steve Tuecke Argonne National Laboratory The University of Chicago Carl Kesselman](https://reader034.vdocument.in/reader034/viewer/2022051315/56649ed25503460f94be267b/html5/thumbnails/13.jpg)
Example: A User Portal
This to be filled in showing the different services protocols APIs etc. used at each layer
![Page 14: The Anatomy of the Grid: An Integrated View of Grid Architecture Ian Foster, Steve Tuecke Argonne National Laboratory The University of Chicago Carl Kesselman](https://reader034.vdocument.in/reader034/viewer/2022051315/56649ed25503460f94be267b/html5/thumbnails/14.jpg)
Example:Network Weather Service
ditto
![Page 15: The Anatomy of the Grid: An Integrated View of Grid Architecture Ian Foster, Steve Tuecke Argonne National Laboratory The University of Chicago Carl Kesselman](https://reader034.vdocument.in/reader034/viewer/2022051315/56649ed25503460f94be267b/html5/thumbnails/15.jpg)
Example: A Data Grid
ditto
![Page 16: The Anatomy of the Grid: An Integrated View of Grid Architecture Ian Foster, Steve Tuecke Argonne National Laboratory The University of Chicago Carl Kesselman](https://reader034.vdocument.in/reader034/viewer/2022051315/56649ed25503460f94be267b/html5/thumbnails/16.jpg)
Layer 1: Fabric
Local access to logical resource May be real component, e.g. CPU, software
module, filesystem May be logical component, e.g. Condor pool
Protocol or API mediated Fabric elements include
SSP, ASP, peer-to-peer, Entropia-like, and enterprise level solutions
![Page 17: The Anatomy of the Grid: An Integrated View of Grid Architecture Ian Foster, Steve Tuecke Argonne National Laboratory The University of Chicago Carl Kesselman](https://reader034.vdocument.in/reader034/viewer/2022051315/56649ed25503460f94be267b/html5/thumbnails/17.jpg)
Layer 2: Connectivity Protocols
Two classes of connectivity protocols underlie all other components
Internet communication Application, transport and internet layer
protocols I.e., transport, routing, DNS, etc.
Security Authentication and delegation Discussed below
![Page 18: The Anatomy of the Grid: An Integrated View of Grid Architecture Ian Foster, Steve Tuecke Argonne National Laboratory The University of Chicago Carl Kesselman](https://reader034.vdocument.in/reader034/viewer/2022051315/56649ed25503460f94be267b/html5/thumbnails/18.jpg)
Connectivity Layer Element:Grid Security Protocols
Protocols TLS with delegation
Services K5ssl, Globus Authorization Service
APIs GSS-API, GAA, SASL, gss_assist
SDKs GlobusIO
![Page 19: The Anatomy of the Grid: An Integrated View of Grid Architecture Ian Foster, Steve Tuecke Argonne National Laboratory The University of Chicago Carl Kesselman](https://reader034.vdocument.in/reader034/viewer/2022051315/56649ed25503460f94be267b/html5/thumbnails/19.jpg)
Layer 3: Resource Protocols
Resource management Storage system access Network quality of service Data transport Resource information
![Page 20: The Anatomy of the Grid: An Integrated View of Grid Architecture Ian Foster, Steve Tuecke Argonne National Laboratory The University of Chicago Carl Kesselman](https://reader034.vdocument.in/reader034/viewer/2022051315/56649ed25503460f94be267b/html5/thumbnails/20.jpg)
Resource Management
Protocols GRAM+GARA (on HTTP)
Resource services Gatekeeper, JobManager, SlotManager
APIs and SDKs GRAM API, Java CoG Kit Client, DUROC
![Page 21: The Anatomy of the Grid: An Integrated View of Grid Architecture Ian Foster, Steve Tuecke Argonne National Laboratory The University of Chicago Carl Kesselman](https://reader034.vdocument.in/reader034/viewer/2022051315/56649ed25503460f94be267b/html5/thumbnails/21.jpg)
Data Transport
Protocols Grid FTP, LDAP for replica catalog
Services FTP, LDAP replica catalog
APIs and SDKs GridFTP client library, copy URL API, replica
catalog access, replica selection
![Page 22: The Anatomy of the Grid: An Integrated View of Grid Architecture Ian Foster, Steve Tuecke Argonne National Laboratory The University of Chicago Carl Kesselman](https://reader034.vdocument.in/reader034/viewer/2022051315/56649ed25503460f94be267b/html5/thumbnails/22.jpg)
Resource Information
Protocol LDAP V3, Registration/Discovery protocol
Service GRIS
APIs & SDKs C API; JNDI, PerlLDAP, ….
![Page 23: The Anatomy of the Grid: An Integrated View of Grid Architecture Ian Foster, Steve Tuecke Argonne National Laboratory The University of Chicago Carl Kesselman](https://reader034.vdocument.in/reader034/viewer/2022051315/56649ed25503460f94be267b/html5/thumbnails/23.jpg)
Layer 4: Grid Protocols
Grid Information Index Services LDAP and Service registration protocol, … GIIS service LDAP APIs and specialized information API
Co-allocation and brokering GRAM (HTTP+RSL) DUROC service DUROC client API, end-to-end reservation
API
![Page 24: The Anatomy of the Grid: An Integrated View of Grid Architecture Ian Foster, Steve Tuecke Argonne National Laboratory The University of Chicago Carl Kesselman](https://reader034.vdocument.in/reader034/viewer/2022051315/56649ed25503460f94be267b/html5/thumbnails/24.jpg)
Grid Protocols (cont)
Online authentication, authorization services HTTP MyProxy, Group policy servers MyProxy API, GAA API,
Many others (e.g.): Resource discovery (Matchmaker) Fault recovery
![Page 25: The Anatomy of the Grid: An Integrated View of Grid Architecture Ian Foster, Steve Tuecke Argonne National Laboratory The University of Chicago Carl Kesselman](https://reader034.vdocument.in/reader034/viewer/2022051315/56649ed25503460f94be267b/html5/thumbnails/25.jpg)
Layer 5: User Protocols
In general, there are many of these, they tend to be one off and not standardized
Examples: Portal toolkits (e.g., Hotpage) Netsolve Cactus framework
![Page 26: The Anatomy of the Grid: An Integrated View of Grid Architecture Ian Foster, Steve Tuecke Argonne National Laboratory The University of Chicago Carl Kesselman](https://reader034.vdocument.in/reader034/viewer/2022051315/56649ed25503460f94be267b/html5/thumbnails/26.jpg)
Role of Standards
Standard APIs: code reuse, portability Certainly a good thing!
Standard protocols enable resource sharing via interoperability of infrastructure The fundamental problem
Note that Grid Forum is already doing this! Scheduling: remote access protocol Data: Grid FTP Security: Grid Security protocols Information: Standard info representations
![Page 27: The Anatomy of the Grid: An Integrated View of Grid Architecture Ian Foster, Steve Tuecke Argonne National Laboratory The University of Chicago Carl Kesselman](https://reader034.vdocument.in/reader034/viewer/2022051315/56649ed25503460f94be267b/html5/thumbnails/27.jpg)
Summary
Grids are about sharing, not making big MPI jobs
Well-defined protocol architecture is essential to understanding and progress Provides a framework for figuring out where
the pieces fit Grid Forum can play an important role by
helping to fill in missing pieces But note that protocol design is not for the
nonspecialist (or fainthearted)!