isgc’2007, taipei, 28-3-2007 grid computing program at peking university in euchinagrid project

ISGC’2007, Taipei, 28-3-2007

Grid Computing Program at Peking University

in EUChinaGRID Project

S. Qian PKU program in EUChinaGRID project 2

ISGC’2007, Taipei, 28-3-2007

Outline• EUChinaGRID project and PKU group

• Grid infrastructure at PKU (School of Physics)

• WP4 (for Grid application) activities at PKU – Biology subgroup: Protein structure analysis

– Physics subgroup: CMS Monte-Carlo simulation and physics analysis

• Main problems and solutions– Networking

– Software installation at Grid sites

• Summary


ISGC’2007, Taipei, 28-3-2007

EUChinaGRID Project

欧中网格项目(More details will be presented by

Dr. Giuseppe ANDRONICO tomorrow)


ISGC’2007, Taipei, 28-3-2007

Project Banner

Interconnection and

Interoperability of Grids

between Europe and

China


ISGC’2007, Taipei, 28-3-2007

Timescale & Budget

• The official start of the project: 1st January 2006.

• Duration: 24 Months

• EU Contribution: 1,299,998 €.

• A total 495 Person Months (325 Funded) of effort


ISGC’2007, Taipei, 28-3-2007

Partners1 Istituto Nazionale di Fisica Nucleare (IT) (coordinator)2 European Organisation for Nuclear Research (CERN) (CH)

3 Università di Roma Tre, Dipartimento di Biologia – Rome (IT)

4 Consortium GARR (IT)

5 Greek Research & Technology Network (GR)

6 Jagiellonian University, Medical College – Cracow (PL)

7 School of Computer Science and Engineering – Beihang University – Beijing (CN)

8 Computer Network Information Center, Chinese Academy of Sciences (CAS) – Beijing (CN)

9 Institute of High Energy Physics, CAS – Beijing (CN)

10 Peking University – Beijing (CN)


ISGC’2007, Taipei, 28-3-2007

Third Parties

1 Academia Sinica Grid Computing Centre (ASGC) – Taipei

2 Università di Roma Tre, Dipartimento di Fisica – Rome (IT)


ISGC’2007, Taipei, 28-3-2007

Targets of the Project

• To foster the creation of a intercontinental eScience community– Training people– Supporting existing and new applications

• To support interoperable infrastructure for grid operations between Europe (EGEE) and China (CNGRID)


ISGC’2007, Taipei, 28-3-2007

WPs (Working Packages)


ISGC’2007, Taipei, 28-3-2007

Work Breakdown Structures

WP Name

1 Project Administrative and technical management （项目行政和技术管理）

2 Network planning and interoperability study （网络规划与互操作研究）

3 Pilot infrastructure operational support （示范基础设施的运作支持）

PKU

4 Applications （应用） PKU

5 Dissemination （宣传推广） PKU


ISGC’2007, Taipei, 28-3-2007

Collaborative tools


ISGC’2007, Taipei, 28-3-2007

Project Web Siteswww.euchinagrid.eu and

www.euchinagrid.cn

(English) (Chinese 中文 )


ISGC’2007, Taipei, 28-3-2007

Infrastructure基础设施


ISGC’2007, Taipei, 28-3-2007

• RB (Resource Broker) + BDII (Berkely Database Information Index) at CNAF (Italy)

• VOMS at CNAF https://voms2.cnaf.infn.it:8443/voms/euchina/

• GridIce （ Grid sites monitoring ） at CNAF• Sites linked:

– Roma 3 (Italy)– CNAF (Italy)– Catania (Italy)– Athens (Greece)– 3 sites in Beijing (CNIC, IHEP and PKU)

What we have already done


ISGC’2007, Taipei, 28-3-2007

Sites Map


ISGC’2007, Taipei, 28-3-2007

Sites Monitoring

BEIJING - PKU


ISGC’2007, Taipei, 28-3-2007

1) April 3-7, 2006 in Beijing, China (done)

2) April 18-21, 2006 in Rome, Italy (done)

3) June 12-16, 2006 at IHEP + Project’s 1st Workshop in Beijing, China (done)

4) September 15-22, 2006 in Rome, Italy + Project’s 1st Conference (done)

5) November 25-26, 2006 at Peking University (done). All Chinese tutors in first time.

6) April 16-20, 2007 at CNIC, Beijing, China

Training Program


ISGC’2007, Taipei, 28-3-2007

Peking University in

EUChinaGRID Project


ISGC’2007, Taipei, 28-3-2007

Subgroups & Personnel

• Biological Research – Protein structure study with NMR (led by Prof. B. XIA ，夏滨 )– C. JIN, Y. FENG, W. GONG, X. GUO, T. WANG.

– To participate in WP4 (4.3)

• High Energy Physics Research – CMS experiment on LHC at CERN (led by Prof. S. QIAN ，钱思进 )– Z. YANG, L. ZHAO, D. MU, S. ZHU, K. KANG

– To participate in WP4 (4.1) and WP3

• Also, both groups are working in WP5


ISGC’2007, Taipei, 28-3-2007

Biology Group


ISGC’2007, Taipei, 28-3-2007

Beijing NuclearMagneticResonance Center

Sponsored by Ministry of Science and Technology, Ministry of Education, Chinese Academy of Science, Chinese Academy of Military Medical Sciences, Managed by Peking University.

National NMR facility established on Nov. 4th, 2002 For research and training in bio-molecular NMR

studies We need to use computer for processing and

analyzing NMR data, for solution structure calculation, and for molecular dynamic simulation.


ISGC’2007, Taipei, 28-3-2007

Key method for obtaining high resolution structure

-----in addition to X-ray Structure

Physiological temperature and condition -----closer to native functional state

Time consuming for structure calculation -----multiple structures and multiple rounds

NMR Spectroscopy


ISGC’2007, Taipei, 28-3-2007

NMR Structure Determination


ISGC’2007, Taipei, 28-3-2007

From Constraints to Structure

Restrained molecular dynamics and simulated annealing


ISGC’2007, Taipei, 28-3-2007

V = Eempirical + Eeffective

with:

Eeffective = ENOE + Etorsion

and Eempirical = Ebond + Eangle + Edihedral + Evdw + Eelectr

• Empirical energy contains all information about the primary structure of the protein and also data about topology and bonds in proteins in general.

• Empirical energy are from experimental data.

Force Field


ISGC’2007, Taipei, 28-3-2007

Energy Minimization


ISGC’2007, Taipei, 28-3-2007

Structure Calculation and Refinement

Normally, 200 structures/round, > 30 rounds.


ISGC’2007, Taipei, 28-3-2007

Recent Structures

1Z6H 2AI61Z7P

2FHM 2HF6 2B9K


ISGC’2007, Taipei, 28-3-2007

Analysis Software

• Protein structure analysis software: Amber.

• Licenses are needed to be granted on all computers involved.

• University Rome III has procured the license and is testing it, hopefully it can be available for use in near future.


ISGC’2007, Taipei, 28-3-2007

PKU-BiologyComputing Need

• By using the Intel 2.4 GHz Xeon CPU

• Each structure needs 4 hours

• Each time to compute 200 structures

• Each protein needs to be computed for 10 times

• Totally 10 proteins to be analyzed

~ 80,000 hours (> 9 years) CPU time > 1TB storage space


ISGC’2007, Taipei, 28-3-2007

Physics Group


ISGC’2007, Taipei, 28-3-2007

Physics Data Analysis for CMS Experiment

CMS group in the Physics School of Peking University has started to use Grid tools to analyze physics data of CMS experiments on LHC at CERN since 9/2005

Huge amount of Monte-Carlo data (from now on) and real data (collected from the end of 2007) shall await for us to analyze

27 km

circumference

LHC completion

date: 2007.11


ISGC’2007, Taipei, 28-3-2007

LHCComputingGrid Model

les.

rob

ert

son

@ce

rn.c

h

physics group

regional group

Tier2

Lab aUni a

Lab c

Uni n

Lab m

Lab b

Uni bUni y

Uni x

Tier3physics

department

Desktop

Germany

Tier 1

USAUK

France

Italy

……….

CERN Tier 1

……….

The LHC Computing

Centre CERN Tier 0


ISGC’2007, Taipei, 28-3-2007

LCG Architecture at PKU

Installed at PKU

(UI)

(SE)

(CE)

(WN)

(SE)

Installed at PKU

(UI) (CE)

@IHEP


ISGC’2007, Taipei, 28-3-2007

Working History• Single J/generation (without background) and reconstruction

by using local computers in 6/2005

• Single J/ study with min-biased background in 7/2005

• Analyzed 500 B0s J/ + events from a DST (Data Summary Tapes)

at CERN in 8/2005

• Analyzed nearly 200,000 B0s events from a DST stored in Italy by using Computing Grid tools from 9/2005 and going on

• Preparing the massive (> 2 millions J/ events) Monte-Carlo simulation


ISGC’2007, Taipei, 28-3-2007

Procedure of Grid Application

The latest procedure via the IHEP LCG Tier-2 facility:

PKU’s UI getsthe results from submit the jobsIHEP’s RB

run the jobs, send the jobs to CEreturn the results to

IHEP’s RB give the jobs to WN

UI (User Interface)@PKU, China

RB (Resource Broker)@IHEP, China

CE (Computing Element)@CNAF, Italy

WN (Work Nodes)@CNAF, Italy


ISGC’2007, Taipei, 28-3-2007

Sample Result

J/psi reconstruction efficiency as a function of PT (both muons’ |eta|<=2.4)

J/

reconstruction efficiency in

CMS

experiment


ISGC’2007, Taipei, 28-3-2007

First CMS Analysis Note by Peking Univ. Group

CMS Analysis Note 2006-094

J μ μ Reconstruction in CMS

Zongchang YANG, Sijin QIANPeking University, China

April 2006 (Revised in November 2006)

AbstractIn this note the J/ψ → μ μ reconstruction was studied in details by using Bs J/ μ μ KK events. The reconstruction efficiencies of J/ψ and decayed di-muons were obtained at various PT and pseudo-rapidity . We also preliminari ly studied the muon trigger efficiency and the J/ψ reconstruction with default L1 and HLT. It was observed that the muon reconstruction efficiency decreases in the case of two decayed muonswith a small or large 3D angu lar separation, which further affect the J/ψ reconstruction efficiency . In an earlier study with the s imple J/ψ even ts, we obtained the upper limits of efficiency and mass resolution for J /ψ offl ine reconstruction in CMS.


ISGC’2007, Taipei, 28-3-2007

PKU-Physics Computing Need

• In 2007, we would wish to generate > 2 million events each for prompt J/Psi and Upsilon + 40% of background events

• For each 1 million events, it needs about 24,000 hours (or 1000 days) of CPU time (for

one P4 Xeon 1.5GHz computer), and about 1.1 TB of storage space.

• In result, we would need ~5600 days (i.e. ~ 18 years) of CPU time & ~6 TB of storage space


ISGC’2007, Taipei, 28-3-2007

Summary of WP3 & WP4 Activities at PKU

• Established a LCG (LHC Computing Grid) Tier-3 site for getting access to the LCG system;

• Used the above system to have analysed a large MC dataset stored at CNAF in Italy, and have produced some analysis results;

• Provided configuration files for CMS collaboration in order to generate >2 million prompt J/ events;

• Installed the CMSSW on EUChinaGrid system (Catania site);

• Preparing the protein structure analysis in Biology group;

• Has estimated the computer and storage resources needed to handle the millions of events for Physics group and to analysis the protein structure in Biology group.


ISGC’2007, Taipei, 28-3-2007

Main Problems

• Availability of biological software (Amber)– Licensing

• Stability of CMS software (CMSSW)– the suitable J/ event generator is still being tested

by CMS collaboration before to be put in production

– HLT (High Level Trigger) software

• Networking– Bandwidth (international traffic is charged by bits)

– University policy (3 levels of gateway)


ISGC’2007, Taipei, 28-3-2007

Networking in PKU

• 3 levels of gateway

– Campus network: no charge, only within campus

– Domestic gateway: minor monthly charge, unlimited traffic

– International gateways:

• Monthly package -- 90 Yuan/month, unlimited traffic, but disconnected every few hours if no activities

• Server gateway -- no interruption, but charged by bits


ISGC’2007, Taipei, 28-3-2007

Solutions

• Use the domestic gateway to connect to IHEP via VPN (Virtual Private Network), then to reach the world through the IHEP’s trunk line.

• Applied and installed the CERNET’s special link to TEIN2. The special cabling was done in 1/2007.

– No charge by bits

– No periodical interruption.


ISGC’2007, Taipei, 28-3-2007

Network Topology Map

The improved route (TEIN2): will upgrade to 2.5 Gbps

The backup route


ISGC’2007, Taipei, 28-3-2007

Summary

• PKU group has set up a very basic Grid site for getting access to the LCG system and for preparing the massive biological protein structure analysis.

• By using this system, we have engaged in some CMS physics study and got some encouraging results.

• Some long standing problems of networking have been finally solved with the TEIN2 connection.

• Much more works are to be done, we must– start the protein structure analysis as soon as the software licence

is granted;

– be fully prepared for the CMS data analysis when LHC’s first proton beam collision at the end of 2007.


ISGC’2007, Taipei, 28-3-2007

Thank you ( 謝謝 ) ！

isgc’2007, taipei, 28-3-2007 grid computing program at peking university in euchinagrid project

Documents