ihep computing center site report shi, jingyan (shi.jingyan@ihep.ac.cn) computing center, ihep
Post on 04-Jan-2016
222 Views
Preview:
TRANSCRIPT
IHEP Computing Center Site IHEP Computing Center Site ReportReport
Shi, Jingyan Shi, Jingyan (shi.jingyan@ihep.ac.cn)(shi.jingyan@ihep.ac.cn)Computing Center,Computing Center,IHEPIHEP
Shi,Jingyan/CC/IHEP 23/4/20 - 2
IHEP Brief IHEP Brief introductionintroduction
• The largest(~1000 stuffs) fundamental The largest(~1000 stuffs) fundamental research center in China with research fieldsresearch center in China with research fields i includncluding: ing: • Particle Physics ExperimentsParticle Physics Experiments
• Cosmic Ray/Astrophysics experimentsCosmic Ray/Astrophysics experiments
• Theoretical PhysicsTheoretical Physics
• ……
• Scientific projects:Scientific projects:• BESIII experiment running on BEPC BESIII experiment running on BEPC • ARGO-YBJ experiment ARGO-YBJ experiment • Daya Bay Daya Bay rreactoreactor n neutrino eutrino eexperiment xperiment • ATLAS, CMS experiment on LHCATLAS, CMS experiment on LHC• AMS, HMXT …AMS, HMXT …
Beijing Spectrum ARGO-YBJ Detector
CC-IHEP at a CC-IHEP at a GlanceGlance
• The Computing Center was created in The Computing Center was created in 1980’s1980’s• Provided computing service to BES, the Provided computing service to BES, the
experiment on BEPCexperiment on BEPC
• Rebuilt in 2005 for the new projects:Rebuilt in 2005 for the new projects:• BES-III on BEPC-IIBES-III on BEPC-II• Tier-2 for ATLAS, CMSTier-2 for ATLAS, CMS• Cosmic ray experiments Cosmic ray experiments
• 35 FTEs, half of them for computing 35 FTEs, half of them for computing facilityfacility
Shi,Jingyan 23/4/20 - 3
Shi,Jingyan 23/4/20 - 4
10Gbit Ethernet
3+ PB disk storage ( Lustre ,dpm, dCache)
More than 8000 CPU cores
5PB tape storage
Login, monitoring, scheduling,AFS,backup…
Outside data farms:LHC, YBJ,DAYABAY
Local data farm
WAN
IHEP Campus IHEP Campus NetworkNetwork
• Network upgrade finished last monthNetwork upgrade finished last month• IPv4/IPv6 enabled for all usersIPv4/IPv6 enabled for all users• Campus wireless network covered Campus wireless network covered
Shi,Jingyan– Kan, Bowen/CC/IHEP 23/4/20 - 5
Shi,Jingyan/CC/IHEP 23/4/20 - 6
Network connectionNetwork connection
DayaBay
ShenZhen
BeijingCSTNet
HongKong
IHEP
USA
GLORIAD 10G
ASGC
TEIN3
IPv4 10G IPv6
BeijingTsinghua
YBJ
EUR.
2.5G 1G
155M
155M
2.5G
Others
EDU.CN2.5G
Computing Computing ResourcesResources
• Local Cluster & Grid SiteLocal Cluster & Grid Site• Cpu:Cpu:
• Work node: More than 8000 cpu/coresWork node: More than 8000 cpu/cores
• Storage:Storage:• 3+PB storage3+PB storage
Lustre for local clusterLustre for local cluster dCache for CMSdCache for CMS DPM for ATLASDPM for ATLAS
• 5PB Tape storage5PB Tape storage
• SchedulerScheduler• Cluster: PBS + MauiCluster: PBS + Maui
Shi,Jingyan 23/4/20 - 7
File system - Lustre
• 4 MDSs, 39 OSSs, 376 OSTs, 800 client nodes, 135 million files
• Lustre Version: 1.8.5 ( upgraded in July) and 1.8.6 (with a new mount point)
• Capacity: 2.8PB • IHEP is considering binding Lustre with
CASTOR 1.7 using the HSM function provided by Lustre 2.x, however, it seems that the HSM feature has not totally landed in stable version of Lustre
Disk ErrorDisk Error• More than 3000 pieces of disks are used for More than 3000 pieces of disks are used for
the storage and 2/3 have expired guarantee the storage and 2/3 have expired guarantee periodperiod• 1TB disks: 1972 pieces1TB disks: 1972 pieces• 2TB disks: 752 pieces2TB disks: 752 pieces• 3TB disks: 432 pieces3TB disks: 432 pieces
• Disk hard error happened once 3-4 days in Disk hard error happened once 3-4 days in averageaverage
• Spare parts are limited, especially for 1TB Spare parts are limited, especially for 1TB disksdisks
• Considering a way to upgrade thoseConsidering a way to upgrade those disks disks smoothlysmoothly
Shi,Jingyan 23/4/20 - 9
Shi,Jingyan– Kan, Bowen/CC/IHEP 23/4/20 - 10
IHEP - CAIHEP - CA• Number of issued certificatesNumber of issued certificates
Last update: Apr20 2012 UTC+8
User Certificate
Host Certificate
Service Certificate
Total
VALID 112 100 0 212
Revoked 254 241 1 496
EXPIRED 344 307 6 657
IHEP PKU HKU BUAA NJNU SEU RSGS
CNIC TSINGHUA
CCNU SDU NNU NJU USTC
• Cooperation OrganizationsCooperation Organizations
CA need to be CA need to be upgradedupgraded
• According to the GFD.125, emailAddress (or According to the GFD.125, emailAddress (or
Email) now MUST NOT be used in subject Email) now MUST NOT be used in subject
or issuer DNs. or issuer DNs.
• IHEP will follow the steps below to remove the IHEP will follow the steps below to remove the
emailAddress.emailAddress.1. Modify the CP/CPS to comply with GFD.1251. Modify the CP/CPS to comply with GFD.125
2. Generate a new key pair (new root CA cert)2. Generate a new key pair (new root CA cert) . Register it to the . Register it to the
IGTF repository so that the new root CA cert and related files will IGTF repository so that the new root CA cert and related files will
be included in the IGTF CA distributionbe included in the IGTF CA distribution
3. 3. Inform the relying parties that the subject DN of CA certs and Inform the relying parties that the subject DN of CA certs and
EE certs would be changed. NEE certs would be changed. New root CA will issue new EE certsew root CA will issue new EE certs
4. 4. Revoke the old CA certificate until all EE certificates issued by Revoke the old CA certificate until all EE certificates issued by
the old CA key will be expiredthe old CA key will be expired
Shi,Jingyan 23/4/20 - 11
BEIJING-LCG2 Site reportBEIJING-LCG2 Site report
BEIJING-LCG2 Site report
Reliability and Availability
Site Operation
• Some new computing resources will be added• 16 old WN will be replaced with new blade
before JUL. 2012• HEPSPEC 2006 will rise up from 8000 to
9600
• EGEE to EGI migration• Testing new system• Grid middleware upgrade• Keep eyes on other sites’ migration.
Question & Thank You
top related