adoption status of the ogsa-bes interface unicore.eu
DESCRIPTION
Morris Riedel, Mohammad Shahbaz Memon … and many others from the UNICORE community Jülich Supercomputing Centre (JSC). Adoption Status of the OGSA-BES interface http://www.unicore.eu. Outline. Architecture Overview OGSA-BES Adoption Overview Job Submission Description Language - PowerPoint PPT PresentationTRANSCRIPT
Adoption Status of theAdoption Status of theOGSA-BES interfaceOGSA-BES interface
http://www.unicore.euhttp://www.unicore.eu
Morris Riedel, Mohammad Shahbaz Memon
… and many others from the UNICORE community
Jülich Supercomputing Centre (JSC)
2http://www.unicore.eu
Architecture Overview OGSA-BES Adoption Overview
Job Submission Description LanguageHigh Performance Computing Basic Profile
OGSA-BES Adoption in Detail OGSA-BES Limitations/Additions Future Work for OGSA-BES
Link to other GIN session: Security Profile References
OutlineOutline
3http://www.unicore.eu
Architecture OverviewArchitecture Overview
4http://www.unicore.eu
UNICOREWS-RF
hostingenvironment
XNJS
ArchitectureArchitecture
IDB
UNICORE Atomic Services
OGSA-*
XACML entity
emerging standardinterfaces
Grid services hosting
job incarnation &authorization
authentication
scientific clientsand applications
parallel scientific jobs of multiple end-users
on target systems
UNICOREWS-RF
hostingenvironment
XNJSIDB
UNICORE Atomic Services
OGSA-*
XACML entity
Gateway
UNICOREWS-RF
hostingenvironment
ServiceRegistry
Gateway
ByteIO
BES
RUS
ByteIO
BES
RUS
XACML XACML
HPC-P
GPEapplication client
command-line client
Eclipse-basedclient
Portal client, e.g. GridSphere
WS-ISOAP JSDL
XUUDB
SAML-VOMS
X.509
UR
JSDL
HPC-P
UR
JSDL
WS-RF
SAML
X.509
Local RMS (e.g. Torque, LL, LSF, etc.)
Target System Interface
Local RMS (e.g. Torque, LL, LSF, etc.)
Target System Interface
DRMAA DRMAA
M. Riedel et al. [1]
R. Ratering et al. [6]
5http://www.unicore.eu
OGSA-BES Adoption OverviewOGSA-BES Adoption Overview
6http://www.unicore.eu
Three Services basicallysubstituting the proprietary services: TargetSystemFactory (TSF) TargetSystemService (TSS) JobManagementService (JMS)
OGSA-BES Factory Service CreateActivity( JSDL )
OGSA-BES Management Service E.g. Allow new submissions or not
Use of UNICORE 6 Backend XNJS Forwarding to TSI on resource
Developed by
OGSA – Basic Execution Services (BES)OGSA – Basic Execution Services (BES)
M.S. Memon et al. [2]
7http://www.unicore.eu
UNICORE 5 Proprietary description:
Abstract Job Object (AJOs) UNICORE 6 Services
Target System Service (TSS)accepts JSDLs
OGSA – Basic Execution Services accepts JSDLs
UNICORE 6 Execution Backend XNJS backend component
natively accept JSDLs JSDLs are transformed
into a proprietary simple protocolto submit job to RMSs (e.g. Torque)
Job Submission Description Language (JSDL)Job Submission Description Language (JSDL)
M. Riedel et al. [1]
8http://www.unicore.eu
(Participated in the Supercomputing 2007 Interoperation demonstrations)
XNJS accept JSDL extensions compliant to ones defineswithin the HPC-BP profile
Service Container provides asecurity handler to accept credentials according toWS-Security Username TokenProfile (part of HPC-BP)
HPC – Basic Profile (HPC-BP)HPC – Basic Profile (HPC-BP)
M.S. Memon et al. [2]
9http://www.unicore.eu
OGSA-BES Adoption in DetailOGSA-BES Adoption in Detail
10http://www.unicore.eu
Support of all three defined portTypes with defined faults Configurations in uas.config configuration file of UNICORE
E.g. initial value whether accept new activities or not(property of BES-Management portType)
Support of BES State Model (and UNICORE state model) Additional States:
BES interface implementation (developer view) Use the xFire SOAP engine Use XMLBeans for Java-XML bindings No problems in creating XMLBeans from BES schemas
(no schema for BES-Activity provided) Sourcecode as open source on UNICORE@SourceForge Execution Management provided by XNJS of UNICORE
OGSA – BES Adoption in Detail (1)OGSA – BES Adoption in Detail (1)
11http://www.unicore.eu
Supports filtered activities during BES-Factory operations… (Filtered activities are activities, which are profiled or associated
with users credentials, and its visibility will be exposed to only the authorized users who created the activity)
Own filter created to support this feature Supported Operations: GetActivityDocuments,
GetActivityStatuses, TerminateActivities UNICORE 6 supports the optional WS-RF Rendering by
adding all WS-RF compliant message exchanges Enables GetResourceProperties, UpdateResourceProperties,… Activity resources are modeled as WS-RF resources, each BES
Activity instance maintains activity document as state Consists of JSDL sent by user, reference to BES-Factory which created
that activity instance, and the current activity status
OGSA – BES Adoption in Detail (2)OGSA – BES Adoption in Detail (2)
12http://www.unicore.eu
Client support in UNICORE UNICORE Rich Client support as additional plug-in
demonstrator UNICORE Command-Line Client (UCC) Support
Interoperability Tests with othe OGSA-BES implementations in the scientific Grid landscape Production Level for e-Infrastructure integration
Only tested with CREAM-BES (SAML-based Security), planned for gLite release – CREAM in certification process
Initial tests with Globus-BES beta (provided by KTH) Interoperablity Testings with other HPC-BP adopters
E.g. NorduGrid ARC implementations, University of Virginia implementations,…
OGSA – BES Adoption in Detail (3)OGSA – BES Adoption in Detail (3)
13http://www.unicore.eu
OGSA-BES Limitations/AdditionsOGSA-BES Limitations/Additions
14http://www.unicore.eu
Additions to the BES-Factory Attributes Document INFOS! GLUE resource property (for specifying admin domain info,
application info, computing resource info) for more information Additions to the BES-Activity portType
(operations are not normatively defined in specification) To support an easy access to single resources (instead of
vectors/arrays as in BES-Factory) operations added In addition, also WS-RF compliant message exchanges work at
this portType – to work with the resource model in UNICORE No aligned security model strong enough for production
High Performance Computing Basic Profile good first step into right direction, but not for production usage in e.g.DEISA
Precise profiled attribute-based AuthZ support required today
OGSA – BES Limitations / Additions (1)OGSA – BES Limitations / Additions (1)
15http://www.unicore.eu
Link to storage added in BES-Activity (out of scope of OGSA-BES – but required in UNICORE) Storage Endpoint to a UNICORE proprietary Storage
Management Service (SMS) – Req. for BES-related activity! SMS Endpoint added to the BES Activity resource properties
Property manages a user space mapping with individual activities SMS can be used to access outcome of computational jobs
File Transfer for data staging issues (not precise enough) SMS link in turn is a factory for managing individual file transfer
functions such as stage-in/out of files for individual jobs File Transfer realized via File Transfer Service (i.e. ByteIO) Data Staging Profile might be interesting but is not complete in-
line with production deployment
OGSA – BES Limitations / Additions (2)OGSA – BES Limitations / Additions (2)
16http://www.unicore.eu
UNICORE 6 basic infrastructure relies on WS-RF Any activity is an instance of the WS-RF resource model
‘Optional WS-RF support’ in specification makes it difficult for UNICORE clients to work with other BES implementations E.g. GetResourceProperties of CREAM-BES fails since it is not
WS-RF compliant ‘Optional WS-RF support’ requires a factory for the factory
WS-RF factory pattern difficult to implement Because factory is not end-user specific instance
OGSA – BES Limitations / Additions (3)OGSA – BES Limitations / Additions (3)
17http://www.unicore.eu
Bottom lines BES is not isolated – it must be well embedded with other
middleware services – but how to standardize/specify Storage is tightly connected with BES File transfer is tightly connected with BES Security Setups are tightly connected with BES
For all these topics are not normative standards defined – only BES/JSDL
More Profiling is required
OGSA – BES Limitations / Additions (4)OGSA – BES Limitations / Additions (4)
18http://www.unicore.eu
Future Work for OGSA-BESFuture Work for OGSA-BES
19http://www.unicore.eu
Enable more functionalities in terms of … data-staging profile, information model GLUE hooks, …
Delegation Scenarios How are third party transfer credentials transferred through the
OGSA-BES interface E.g. data stage-in is performed using GridFTP (requiring
credentials – is not an implementation issue) SAML delegation vs. Proxy delegation Link to GIN Security
Profile Session More production-oriented security profiles aligned with BES
Attribute-based Authorization, where credentials transported? E.g. WS-Security Message exchanges in SOAP header
Link to GIN Security Profile Session
Future Work for OGSA-BESFuture Work for OGSA-BES
20http://www.unicore.eu
ReferencesReferences
21http://www.unicore.eu
[1] M. Riedel, B. Schuller, D. Mallmann, R. Menday, A. Streit, B. Tweddell, M.S. Memon, A.S. Memon, B. Demuth, Th. Lippert, D. Snelling, S. van den Berghe, V. Li, M. Drescher, A. Geiger, G. Ohme, K. Benedyczak, P. Bala, R. Ratering, A. Lukichev Web Services Interfaces and Open Standards Integration into the European UNICORE 6 Grid Middleware Proceedings of 2007 Middleware for Web Services (MWS 2007) Workshop at 11th International IEEE EDOC Conference "The Enterprise Computing Conference", 2007, Annapolis, Maryland, USA, to appear
[2] M.S. Memon, A.S. Memon, M. Riedel, B. Schuller, D. Mallmann, B. Tweddell, A. Streit, S. van den Berghe, D. Snelling, V. Li, M. Marzolla, P. Andreetto Enhanced Resource Management Capabilities using Standardized Job Management and Data Access Interfaces within UNICORE Grids Proceedings of 3rd Workshop on Scheduling and Resource Management for Parallel and Distributed Systems SRMPDS 2007, ICPADS'07 - The 13th International Conference on Parallel and Distributed Systems Hsinchu, Taiwan, December, 2007, to appear
[3] M. Riedel, R. Menday, A. Streit, and P. BalaA DRMAA-based Target System Interface Framework for UNICOREProceedings of Second International Workshop on Scheduling and Resource Management for Parallel and Distributed Systems (SRMPDS’06) at Twelfth International Conference on Parallel and Distributed Systems (ICPADS’06), Minneapolis, USA, IEEE Computer Society Press, pages 133 - 138
References (I)References (I)
22http://www.unicore.eu
[4] W. Frings, M. Riedel, A. Streit, D. Mallmann, S. van den Berghe, D. Snelling, and V. LiLLview: User-Level Monitoring in Computational Grids and e-Science Infrastructures.In Proc. of German e-Science Conference 2007, Baden-Baden, Germany, Online-Publication
[5] V. Venturi, M. Riedel, A.S. Memon, M.S. Memon, F. Stagni, B. Schuller, D. Mallmann, B. Tweddell, A. Gianoli, V. Ciaschini, S. van de Berghe, D. Snelling, A. StreitUsing SAML-based VOMS for Authorization within Web Services-based UNICORE Grids.Proceedings of 3rd UNICORE Summit 2007 in conjunction with EuroPar 2007, Rennes, France, to appear
[6] R. Ratering, A. Lukichev, M. Riedel, D. Mallmann, A. Vanni, C. Cacciari, S. Lanzarini, P. Bala, K. Benedyczak, M. Borcz, R. Kluszcynski, and G. Ohme,GridBeans: Supporting e-Science and Grid Applications.In Proc. of the 2nd IEEE International Conference on e-Science and Grid-Computing (e-Science 2006), IEEE Computer Society Press, Amsterdam, NL, December 4-6, 2006, ISBN: 0-7695-2734-5, proceedings on CD
References (II)References (II)