WLCG Grid Deployment Board

CERN, 14 May 2008

Storage UpdateStorage Update

Flavia Donno

CERN/IT

WLCG Grid Deployment Board, CERN 13 May 2008 2

OverviewOverview

Summary of the discussions on the SRM v2.2 Addendum to the Usage Agreement

Some questions for the experiments

WLCG Grid Deployment Board, CERN 13 May 2008 3

The experiments requirementsThe experiments requirements

The goal of the SRM v2.2 Addendum is

to provide answers to the following Main questions :

What kind of spaces (T1D0, T1D1, T0D1) Space protection Space selection Pre-staging Space usage statistics Tape optimization (reducing the number of mount operations, etc.)

Main dates: 19 May: technical agreement among the developers 23 May: experiment agreement. Document submitted to a few site

representatives for comments. 2 June: document finalization with time estimation for implementation. 10 June: submission to the MB for approval. Start of the implementation

phase.

WLCG Grid Deployment Board, CERN 13 May 2008 4

The document and validationThe document and validation

2 main parts: A detailed description of a full solution implementation independent

Implementation plans per storage solution are also detailed with dates for each of the features.

A summary of what can be achieved by the end of the year per implementation

For the partial and full solutions a test suite will be ready for the validation of the implementations

Most probably also a draft plan for certification and deployment will be included in the document.

WLCG Grid Deployment Board, CERN 13 May 2008 5

The MoU AddendumThe MoU Addendum Spaces, files, copies and TURLs are defined The behavior of the implementations for a set of

operations has been clarified. srmRm it is an operation on namespace only. srmLs must return the arrayOfSpaceTokens. srmPurgeFromSpace removes disk copies only. srmBringOnline can be used to bring a file in a different space srmPrepareToPut/PutDone does not allow to specify for how long the

created copy stays on disk. This is system specific. It is not possible to release copies created via a Put cycle or via an

srmCopy operation. At the moment srmCopy does not allow to pass tokens and destination

copy lifetime. We will use a specific SRM structure to pass this info. Clients would need to be modified to check this structure.

srmReleaseFiles must be properly implemented to allow releasing of files without specifying the request ID.

WLCG Grid Deployment Board, CERN 13 May 2008 6

The MoU AddendumThe MoU Addendum

Clients can now specify tokens for space selection

on srmPrepareToGet, srmBringOnline, srmCopy srmChangeSpaceForFiles is not mandatory

Space Tokens can be specified on Get The concept of file primary copy is not used in WLCG No tape transitions are allowed in WLCG

srmLs fileLocality parameter reflects the status of all copies. If there is a nearline copy of a file in T1D0 and another online copy in T0D1,

the status of the file is NEARLINE_AND_ONLINE

To optimize tape usage the structure TExtraInfo can be used to pass additional implementation specific info that are ignored by other implementations

WLCG Grid Deployment Board, CERN 13 May 2008 7

The MoU AddendumThe MoU Addendum Access Control Lists and a set of allowed

operations are defined on spaces.

Administrative interface. Read/Write/Stage/Purge are mandatory operations Release/Update/Query/Modify-ACLs are optional More granularity with implementation specific features

Permissions on spaces are expressed in terms of DNs and/or VOMS FQANs. Optional FQAN wildcards.

ACLs can be positive or negative.fqan:dteam/Role=production:RSWQP:ALLOWfqan:dteam/Role=lcgamin:DURWSPQM:ALLOWfqan:dteam RSQ:ALLOW fqan:EVERYONE:DURWSPQM:DENY

NFSv4-type ACLs with either POSIX or NFSv4 matching algorithm Definition of ACLs modification operations

WLCG Grid Deployment Board, CERN 13 May 2008 8

Questions for expsQuestions for exps Comments on pinning copies ?

Pinning on Put is implementation specific Pinning will remain soft for some implementations

The space after a file removal operation will not be made available immediately

srmLs fileLocality returns the status of all copies srmReleaseFiles with no requestID

Is this needed ? Please, provide valid use cases. Sign with the name of the experiment and person providing usecase.

Comments on protecting spaces Is this enough ? It will slow down operations ? Wildcards on FQAN

Anything else ?

WLCG Grid Deployment Board

CERN, 14 May 2008

Thank YouThank You