cms tier 1 at jinr v.v. korenkov for jinr cms tier-1 team jinr
DESCRIPTION
CMS Tier 1 at JINR V.V. Korenkov for JINR CMS Tier-1 Team JINR. XXIV International Symposium on Nuclear Electronics & Computing, NEC2013 2013, September 13. Outline. CMS Grid structure role of Tier-1s CMS Tier-1s CMS Tier-1 in Dubna History and Motivations (Why Dubna?) - PowerPoint PPT PresentationTRANSCRIPT
CMS Tier 1 at JINRV.V. Korenkov
for JINR CMS Tier-1 TeamJINR
XXIV International Symposium on Nuclear Electronics & Computing, NEC2013
2013, September 13
2Outline
• CMS Grid structure– role of Tier-1s– CMS Tier-1s
• CMS Tier-1 in Dubna– History and Motivations (Why Dubna?)– Network infrastructure– Infrastructure and Resources– Services and Readiness– Staffing– Milestones
• Conclusions
3
CMS Grid Structure
Tier Structure of GRID Distributed Computing:
Tier-0/Tier-1/Tier-2
Tier-0 (CERN):• accepts data from the CMS Online Data Acquisition and Trigger System
• archives RAW data • the first pass of reconstruction and performs Prompt Calibration
• data distribution to Tier-1Tier-1 (11 centers):• receives a data from the Tier-0 • data processing (re-reconstruction, skimming , calibration etc)• distributes data and MC to the other Tier-1 and Tier-2• secure storage and redistribution for data and MCTier-2 (>200 centers):• simulation• user physics analysis
5
CMS Tier-1 in Dubna
6
Tier1 centerTier1 center
The Federal Target Programme ProjectThe Federal Target Programme Project: : «Creation of the «Creation of the automated system of data processing for experiments at automated system of data processing for experiments at the LHC of Tier-1 level and maintenance of Grid services the LHC of Tier-1 level and maintenance of Grid services for a distributed analysis of these data» for a distributed analysis of these data»
Duration:Duration: 2011 – 20132011 – 2013
March 2011 - Proposal to create the LCG Tier1 center in Russia (official letter by Minister of Science (official letter by Minister of Science and Education of Russia A. Fursenko has been sent to and Education of Russia A. Fursenko has been sent to CERN DG R. Heuer):CERN DG R. Heuer):
NRC KI for ALICE, ATLAS, and LHC-BLIT JINR (Dubna) for the CMS experiment
Full resources - in 2014 to meet the start of Full resources - in 2014 to meet the start of next working LHC session.next working LHC session.
September 2012 – Proposal was reviewed by WLCG OB and JINR and NRC KI Tier1 sites were accepted as a new “Associate Tier1”
7
Why in Russia?Why Dubna?
• In frames of the RDIG project (a participant of the WLCG/EGEE projects), a grid-infrastructure excepted by LHC experiments has been successfully launched as a distributed cluster RuTier2 (Russian Tier2) and JINR cluster JINR-LCG2 is the main one in RDIG as to its performance.
8
JINR-LCG2 JINR-LCG2
~40% of CPU time in RDIG for 2011-
2013
9JINR Central Information and Computing Complex (CICC)
JINR-LCG2 Normalised CPU time by LHC VOs. January - September 2013.
More than 3 million jobs runTotal normalised CPU time – 20 346 183 kSI2K-hours
Grid users (WLCG)
Local JINR users (no grid)
Jobs run by JINR Laboratories and experiments executed at CICC January - September 2013.
BLTP 8,99%
MPD 39,12%
LRB 1,00% LIT 1,61%COMPASS 13,23%
DLNP 3,84%
FLNP 0,40%
VBLHEP 3,57%
PANDA 27,39%
LBES 0,86%
http://accounting.egi.eu/http://lit.jinr.ru/view.php?var1=comp&var2=ccic&lang=rus&menu=ccic/menu&file=ccic/statistic/stat-2013
10CMS Computing at JINR
the first RDMS CMS web-server (in 1996);
full-scale CMS software infrastructure support since 1997
JINR CMS Tier2 center is one of the most reliable and productive CMS Tier2 centers worldwide (in the top ten best) the most powerful RDMS CMS Tier2 center
CMS Regional Operation Center are functioning in JINR since 2009
The core services needed for WLCG Tier-1 are computing service, a storage service, information service.
The primary Tier-1 tasks can be divided into recording raw data from CERN and storing them on tape; recording processed data from CERN and storing them on disk; providing data to other Tier-1 / Tier-2; reprocessing raw data; event simulation calculations.
111111
Russia Normalized CPU time per SITE and VO (2012-2013)
All VORussia - 409,249,900 JINR - 183,008,044
CMSRussia - 112,025,416JINR - 67,938,700 (61%)
12
Network infrastructure
13 The Core of LHC Networking: LHCOPN and Partners
13
15
Infrastructureand
Facilities
16
17JINR CMS Tier-1 progress
● Disk & server installation and tests: done● Tape system installation: done● Organization of network infrastructure
and connectivity to CERN via GEANT: done
● Registration in GOC DB and APEL: done● Tests of WLCG services via Nagios: done
20132013 20142014 20152015 20162016
CPU (HEPSpec06)CPU (HEPSpec06)
Number of coreNumber of core
2880028800
24002400
5760057600
48004800
69120
5760
82944
6912
Disk (Terabytes)Disk (Terabytes) 35003500 45004500 5400 6480
Tape (Terabytes)Tape (Terabytes) 57005700 80008000 9600 10520
Link CERN-JINRLink CERN-JINR 1010 1010 40 40
18
18
JINR monitoringJINR monitoring
Network monitoring information system -
more than 423 network nodes are in round-the-
clock monitoring
19
Servicesand
Readiness
20CMS-specific activity
● Currently commissioning Tier-1 resource for Currently commissioning Tier-1 resource for CMSCMS::– Local Tests of CMS VO-services and CMS SW Local Tests of CMS VO-services and CMS SW – The PhEDEx LoadTest (tests of data transfer links)The PhEDEx LoadTest (tests of data transfer links)– Job Robot Tests (or Job Robot Tests (or tests via HammerCloudtests via HammerCloud))– Long-running CPU intensive jobsLong-running CPU intensive jobs– Long-running I/O intensive jobsLong-running I/O intensive jobs
● PHDEDX tPHDEDX transferred of ransferred of RAW RAW input data to our storage input data to our storage element with transfer efficiency around 90%element with transfer efficiency around 90%
● Prepared services and data storage for the reprocessing of Prepared services and data storage for the reprocessing of 20120122 88 TeV reprocessing TeV reprocessing
21CMS Tier-1 Readiness
22CMS Tier-1 in Dashborad
Data transfer link to CERN
23/982323
Frames for Grid cooperation of JINRFrames for Grid cooperation of JINR Worldwide LHC Computing Grid (WLCG) Enabling Grids for E-sciencE (EGEE) - Now is EGI-InSPIREEGI-InSPIRE RDIG Development CERN-RFBR project “Grid Monitoring from VO perspective” BMBF grant “Development of the grid-infrastructure and tools to provide joint
investigations performed with participation of JINR and German research centers” “Development of grid segment for the LHC experiments” was supported in frames
of JINR-South Africa cooperation agreement; Development of grid segment at Cairo University and its integration to the JINR Development of grid segment at Cairo University and its integration to the JINR
GridEdu infrastructureGridEdu infrastructure JINR - FZU AS Czech Republic Project “The grid for the physics experiments”JINR - FZU AS Czech Republic Project “The grid for the physics experiments” NASU-RFBR project “Development and support of LIT JINR and NSC KIPT grid-NASU-RFBR project “Development and support of LIT JINR and NSC KIPT grid-
infrastructures for distributed CMS data processing of the LHC operation”infrastructures for distributed CMS data processing of the LHC operation” JINR-Romania cooperation Hulubei-Meshcheryakov programme JINR-Moldova cooperation (MD-GRID, RENAM) JINR-Mongolia cooperation (Mongol-Grid)
24
ROLE FTE
Administrative 1.5
Network support 2
Engineering Infrastructure
2.5
Hardware support 3
Core software and WLCG middleware
4.5
CMS Services 3.5
Total 17
Korenkov V.
Dolbilov A.
Shmatov S.
Trofimov V.
Mitsyn V.
Staffing
25
Milestones
26Objective Target date
Presentation the Execution Plan to WLCG OB Sep 2012
Prototype
Disk & Servers installation and tests Oct 2012
Tape system installation Nov 2012
Organization of network infrastructure and connectivity to CERN via GEANT (2 Gb) Nov 2012
WLCG OPN integration (2 Gb) and JINR-T1 registration in GOCDB including integration with the APEL accounting system
Dec 2012
M1 Dec 2012
LHC OPN functional tests (2 Gb) May 2013
Test of WLCG and CMS services (2 Gb LHCOPN) May 2013
Test of tape system at JINR: data transfers from CERN to JINR (using 2 Gb LHC OPN) May 2013
Test of publishing accounting data May 2013
Definition of Tier 2 sites support May 2013
Connectivity to CERN 10 Gb Jul 2013
M2 Jul 2013
LHC OPN functional tests (10 Gb) Aug 2013
Test of tape system at JINR: data transfers from CERN to JINR (using 10 Gb LHC OPN) Aug 2013
Upgrade of tape, disk and CPU capacity at JINR Nov 2013
M3 Nov 2013
85% of the job capacity running for at least 2 months
Storage availability > 98% (functional tests) for at least 2 months
Running with > 98% Availabilities & Reliabilities for at least 30 days
WLCG MoU as an associate Tier-1 center Feb 2014
Disk & Tape & Servers upgrade Oct 2014
M4 Dec 2014
Milestones of the JINR CMS Tier-1 Deployment and Commissioning
27 Main tasks for next yearsMain tasks for next years
Engineering infrastructure (system of Engineering infrastructure (system of uninterrupted power supply and climate-uninterrupted power supply and climate-control)control)
High-speed reliable network infrastructure High-speed reliable network infrastructure with the allocated reserved channel to CERN with the allocated reserved channel to CERN (LHCOPN)(LHCOPN)
Computing system and storage system on the Computing system and storage system on the basis of disk arrays and tape libraries of high basis of disk arrays and tape libraries of high capacitycapacity
100% reliability and availability. 100% reliability and availability.
Lyon/CCIN2P3Barcelona/PIC
De-FZK
US-FNAL
Ca-TRIUMF
NDGF
CERNUS-BNL
UK-RAL
Taipei/ASGC
26 June 2009
Amsterdam/NIKHEF-SARA
Bologna/CNAF
Russia:NRC KI
JINR
29/98
The 6th International Conference "Distributed Computing and Grid-technologies in Science
and Education" (GRID’2014)Dubna, 30 June-5 July 2014
GRID’2012 Conference
22 countries, 256 participants, 40 Universities and
Institutes from Russia,
31 Plenary, 89 Section talks
30Conclusions
In 2012-2013 CMS Tier1 prototype was In 2012-2013 CMS Tier1 prototype was created in Dubnacreated in Dubna Disk & server installation and testsDisk & server installation and tests Prototype tape system installation and testsPrototype tape system installation and tests Organization of network infrastructure and Organization of network infrastructure and
connectivity to CERN via GEANTconnectivity to CERN via GEANT Registration in GOC DB and APELRegistration in GOC DB and APEL Tests of WLCG services via NagiosTests of WLCG services via Nagios CMS-specific testsCMS-specific tests Commissioning data transfer links (T0-T1, T1-T1, Commissioning data transfer links (T0-T1, T1-T1,
T1-T2) in progressT1-T2) in progress
We expect to meet the start of next LHC run We expect to meet the start of next LHC run with full resources required (for the end of with full resources required (for the end of 2014)2014)