dpm status and plans

CERN IT Department

CH-1211 Genève 23

Switzerlandwww.cern.ch/

it

DPM status and plans

David Smith

CERN, IT-DM-SGT

Pre-GDB, Grid Storage Services

11 November 2008

CERN IT Department

CH-1211 Genève 23


it

InternetServices

Overview

• DPM– Current status– Forthcoming release– Plans

• Summary

DPM status and plans - 2

CERN IT Department

CH-1211 Genève 23


it

InternetServices

DPM current status

• DPM 1.6.11-3 is the last stable release on SLC4 (since 18 Aug)

• Issues are reported and handled viaGGUS, Savannah, support and user-forum mailing lists

• DPM is used by about 187 sites in LCG– Current LCG statistics:

https://twiki.cern.ch/twiki/bin/view/LCG/GSSDSRMv2DeploymentMonitoring


CERN IT Department

CH-1211 Genève 23


it

InternetServices

DPM v1.7.0

The next release is expected to be v1.7.0 and is currently in certification. Changes include:

• srmCopy has been fully implemented

• Additions for the WLCG SRM MoU addendum:

– write permission to a space can be limited to multiple groups

– SRM2.2 srmReleaseFiles can be used without a requestToken

– srmLs can return the list of spaces a given file is in

• A new dpm-listspaces command line tool which also serves as a information provider has been added (a contribution from Michel Jouvin)

• DPM retains space token when the admin drains a file from a filesystem (as a part of maintenance)


CERN IT Department

CH-1211 Genève 23


it

InternetServices

DPM v1.7.0

• Added ADLER32 and CRC32 to the supported checksum types for the gridftp CKSM command

– Server side checksum calculation of a file already in the DPM

• New nameserver API calls to set or retrieve checksums

• Added SURL cleanup after expiry or abort in some circumstances

• Made some changes in preparation for SLC5

• Some more internal logging for space capacity updates

• Version 1.7.0 will require an update of the dpm_db database schema

– YAIM will do the update automatically. As usual a database backup beforehand is recommended.


CERN IT Department

CH-1211 Genève 23


it

InternetServices

DPM information provider

• Version 1.7.0 will use dpm-listspaces to provide more information about spaces

• A gLite patch was also created to add dpm-listspaces to the current DPM 1.6.11-3 version

– Entered certification before the DPM 1.7.0 release, may be available as a gLite update before the new DPM release

• Updates glite-yaim-dpm, glite-info-dynamic-dpm, DPM-interfaces

• Adds glite-info-provider-service

– Will require YAIM to be rerun to setup the new providers but no changes in site-info.def needed


CERN IT Department

CH-1211 Genève 23


it

InternetServices

DPM plans

• A more friendly DPM python interface has been developed

– Not yet released, decided to add to gLite as an entirely new package separate from the main DPM bundle

For DPM the following are areas of interest

• Enhance xrootd support

– Pass xrootd client authentication details to the DPM for authorization decisions

• Quota system

• Further checksum support

– Calculation on demand is already in gridftp2


CERN IT Department

CH-1211 Genève 23


it

InternetServices

DPM plans

• Monitoring interface

– To provide feedback on the state of the system

• Tool for consistency check and repair

– Both internal DPM structures and possible namespace verification by users

• Enhance filesystem selection

• Investigate a shared filesystem interface (e.g. NFS 4.1)

– Input from the T1/T2 analysis working group will be used to decide direction

• Resource limits

– Service resource usage and disk server I/O limits

• Compliance with the gLite logging recommendations

– In particular logging via syslog for authentication events

• Must also remain alert to the issues arising from sites and experiment requests


CERN IT Department

CH-1211 Genève 23


it

InternetServices

Summary

• New release of DPM is currently in certification– There may be an update of the information

provider for the current DPM version beforehand

• Have shown areas of interest for DPM development– But must balance with the requests and issues

arising in the significant deployment base


dpm status and plans

Documents