Download - Storage Update
WLCG Grid Deployment Board
CERN, 14 May 2008
Storage UpdateStorage Update
Flavia Donno
CERN/IT
WLCG Grid Deployment Board, CERN 13 May 2008 2
OverviewOverview
Summary of the discussions on the SRM v2.2 Addendum to the Usage Agreement
Some questions for the experiments
WLCG Grid Deployment Board, CERN 13 May 2008 3
The experiments requirementsThe experiments requirements
The goal of the SRM v2.2 Addendum is
to provide answers to the following Main questions :
What kind of spaces (T1D0, T1D1, T0D1) Space protection Space selection Pre-staging Space usage statistics Tape optimization (reducing the number of mount operations, etc.)
Main dates: 19 May: technical agreement among the developers 23 May: experiment agreement. Document submitted to a few site
representatives for comments. 2 June: document finalization with time estimation for implementation. 10 June: submission to the MB for approval. Start of the implementation
phase.
WLCG Grid Deployment Board, CERN 13 May 2008 4
The document and validationThe document and validation
2 main parts: A detailed description of a full solution implementation independent
Implementation plans per storage solution are also detailed with dates for each of the features.
A summary of what can be achieved by the end of the year per implementation
For the partial and full solutions a test suite will be ready for the validation of the implementations
Most probably also a draft plan for certification and deployment will be included in the document.
WLCG Grid Deployment Board, CERN 13 May 2008 5
The MoU AddendumThe MoU Addendum Spaces, files, copies and TURLs are defined The behavior of the implementations for a set of
operations has been clarified. srmRm it is an operation on namespace only. srmLs must return the arrayOfSpaceTokens. srmPurgeFromSpace removes disk copies only. srmBringOnline can be used to bring a file in a different space srmPrepareToPut/PutDone does not allow to specify for how long the
created copy stays on disk. This is system specific. It is not possible to release copies created via a Put cycle or via an
srmCopy operation. At the moment srmCopy does not allow to pass tokens and destination
copy lifetime. We will use a specific SRM structure to pass this info. Clients would need to be modified to check this structure.
srmReleaseFiles must be properly implemented to allow releasing of files without specifying the request ID.
WLCG Grid Deployment Board, CERN 13 May 2008 6
The MoU AddendumThe MoU Addendum
Clients can now specify tokens for space selection
on srmPrepareToGet, srmBringOnline, srmCopy srmChangeSpaceForFiles is not mandatory
Space Tokens can be specified on Get The concept of file primary copy is not used in WLCG No tape transitions are allowed in WLCG
srmLs fileLocality parameter reflects the status of all copies. If there is a nearline copy of a file in T1D0 and another online copy in T0D1,
the status of the file is NEARLINE_AND_ONLINE
To optimize tape usage the structure TExtraInfo can be used to pass additional implementation specific info that are ignored by other implementations
WLCG Grid Deployment Board, CERN 13 May 2008 7
The MoU AddendumThe MoU Addendum Access Control Lists and a set of allowed
operations are defined on spaces.
Administrative interface. Read/Write/Stage/Purge are mandatory operations Release/Update/Query/Modify-ACLs are optional More granularity with implementation specific features
Permissions on spaces are expressed in terms of DNs and/or VOMS FQANs. Optional FQAN wildcards.
ACLs can be positive or negative.fqan:dteam/Role=production:RSWQP:ALLOWfqan:dteam/Role=lcgamin:DURWSPQM:ALLOWfqan:dteam RSQ:ALLOW fqan:EVERYONE:DURWSPQM:DENY
NFSv4-type ACLs with either POSIX or NFSv4 matching algorithm Definition of ACLs modification operations
WLCG Grid Deployment Board, CERN 13 May 2008 8
Questions for expsQuestions for exps Comments on pinning copies ?
Pinning on Put is implementation specific Pinning will remain soft for some implementations
The space after a file removal operation will not be made available immediately
srmLs fileLocality returns the status of all copies srmReleaseFiles with no requestID
Is this needed ? Please, provide valid use cases. Sign with the name of the experiment and person providing usecase.
Comments on protecting spaces Is this enough ? It will slow down operations ? Wildcards on FQAN
Anything else ?
WLCG Grid Deployment Board
CERN, 14 May 2008
Thank YouThank You