interoperability demos and atlas applications luca vaccarossa (infn - milano)

November 21, 2002 Luca Vaccarossa INFN - Milano 1

Interoperability Demos and Atlas applications

Luca Vaccarossa (INFN - Milano)ATLAS Software Workshop21/11/2002 CERN - Genève

http://www.pparc.ac.uk/

http://www.cern.ch/


Layout of Talk

• Goals of EU-US joint Testbed• Testbed Infrastructure• Screenshots of Monitoring Tools• Distribution of Atlas Software• Description of the Demo Submission• Conclusions


Joint EU-US grid demosIST2002 4-6 November, Copenaghen

SC2002 16-22 November, Baltimore

Goals of WorldGrid Testbed :

• Interoperability between grid domains for applications submitted by users from different virtual organizations

• The testbed should provide a grid-integrated infrastructure of middleware software transparent to the higher layers so that applications on EU and US could run as they did before the integration, without major modifications.


Joint EU-US grid demos

• The testbed should be heterogeneous supporting various flavors of RH and various flavors of the basic Globus services declared to be interoperable.

• Controlled use of shared resources subject to agreed policy

• Integrated use of heterogeneous resources from iVDGL, EDG and CrossGrid testbed domains


WorldGridShared Global Resources, Interoperability, Policy


DataTAG WP4 WorldGrid Team• DataTAG WP4

• Main Subject:• Interoperability between EU and US Grids services from DataGrid, GriPhyN, PPDG and in collaboration with iVDGL, for the HEP applications.

• WP Manager• A. Ghiselli INFN -CNAF

• DataTAG WP4 Task Force for Demos Testbed Setup

• Coordinated by Flavia Donno (CERN)• David Rebatto INFN - Milano• Sergio Fantinel INFN - LNL• Gennaro Tortone INFN - Napoli• Federica Fanzago INFN - LNL• Marco Verlato INFN - Padova• Vincenzo Ciaschini INFN - CNAF• Luca Vaccarossa INFN - Milano


WorldGrid Testbed Infrastructure

• Two EDG 1.2.3 Resource Broker available (Milano

and Pisa), each one with a related EDG 1.2.3

Information Index (Milano and Pisa) running MDS 2.2

• One central EDG Resource Broker “Glue-aware”

(CNAF-Bologna)

• EDG and CG Computing and Storage Elements

with the EDG and GLUE schema and Information

Providers

• VDT servers serving as Computing and Storage

Elements with the EDG and GLUE schema and

Information Providers


WorldGrid Testbed Infrastructure

• PBS, LSF and Condor-G as LRM but only PBS and

LSF used with RB.

• Multiple UI running EU and US portals such as

Genius and Grappa (Pisa, ANL, Padova, CNAF-

Bologna)


UIUIGenius/GrappaGenius/Grappa

RB/JSS

II/MDS2.2

Replica Catalogue

VDT Servers

CE

SE

World-Grid InfrastructureWorld-Grid Infrastructure


The WorldGrid Testbed


Monitoring Tools

• Nagios – Sergio Fantinel, Gennaro Tortone

(DataTAG)

• Ganglia - Catalin Dumitrescu (University

of Chicago)

• VO Centric


Nagios Monitoring


Ganglia Monitoring


Distribution of Atlas SW• Starting from RPM distribution for Atlas software,

Version 3.2.1. Release 2 from EDG repository● Atlas kit 3.1.2 rpm pacman-ized (Atlas-

kit.pacman) for US sites

● Included additional Components needed at run-time missing in many US sites (LessTif, zsh)

● Distributed as part of ScienceGrid Cache

● Installation check package Atlas-kit-verify


US Sites EU Sites

Gainesville/FL (VDT) Milano/Italy (EDG)

Milwaukee/WI (VDT) Padova/Italy (EDG)

Batavia/IL (VDT) Geneve/Switzerland (EDG)

Bloomington/IN (VDT) Valencia/Spain (CG)

Karlsruhe/Germany (CG)

Lisbone/Portugal (CG)

Sites where Atlas- kit has been installed

Distribution of Atlas SW


Demo Submission ScriptExecutable = "/usr/bin/env";

Arguments = "zsh prod.dc1 00001";

StdOutput = " dc1.002000.test.00001.hlt.pythia_jet_17.log";

StdError = "dc1.002000.test.00001.hlt.pythia_jet_17.err";

InputSandbox={"prod.dc1","dc1.002000.evgen.0001.hlt.pythi

a_jet_17.root","plot.kumac"};

OutputSandbox =

{"dc1.002000.test.00001.hlt.pythia_jet_17.log",

"dc1.002000.test.00001.hlt.pythia_jet_17.his",

"dc1.002000.test.00001.hlt.pythia_jet_17.err","plot.kumac"};

Rank = other.FreeCPUs;

Requirements =

IsMember(other.RunTimeEnvironment,"ATLAS-3.2.1");


Demo Submission Script

• Simulation of 10 events

• Based on DC1 Grid script

• Datafile of 1.7 MB sent trough RB InputSandbox

• Requirements = (RunTimeEnvironment, "ATLAS-

3.2.1”)

• Rank FreeCPUs

• Trough Genius Portal


•Matchmaking by EDG RB

# dg-job-list-match dc1_part1.jdl

The following CE(s) matching your job requirements have been

found:

- grid001.mi.infn.it:2119/jobmanager-pbs-workq

- grid011.pd.infn.it:2119/jobmanager-lsf-datatag

- ufdemo.phys.ufl.edu:2119/jobmanager-pbs-Long

- lxshare0229.cern.ch:2119/jobmanager-pbs-short

- lxshare0229.cern.ch:2119/jobmanager-pbs-medium

- lxshare0229.cern.ch:2119/jobmanager-pbs-long

- - worldgrid.iu.edu:2119/jobmanager-pbs-Long20

- worldgrid.iu.edu:2119/jobmanager-pbs-Long

- ricci.phys.uwm.edu:2119/jobmanager-pbs-Long

- nest.phys.uwm.edu:2119/jobmanager-pbs-Long

- lngrid02.lip.pt:2119/jobmanager-pbs-qgrid

- handsome.dhcp.fnal.gov:2119/jobmanager-pbs-Long

- bee001.ific.uv.es:2119/jobmanager-pbs-qgrid


GENIUS (Grid Enabled web eNvironment for

site Independent User job Submission)

[https://genius.ct.infn.it]

OS & Net services

Basic Services

High level GRID middleware

ALICE ATLAS CMS LHCbApplications’specific layer

Other apps

GLOBUS

toolkit

DataGRID architectur

e

GENIUS web portal

Demo Submission


Demo Submission

• Job executed at remote Cluster

• dg-job-status retrieves job status (Ready,

Scheduled, etc.)

• when Status = OutputReady

dg-job-get-output JobID

• Output sandbox files for the job is retrieved and

stored in a local directory


Demo Submission

Job Output:

• Standard output & error

• Histogram file of 190 KB

Interactive Session

• Genius Interactive Services

• PAW


Conclusions

• The demo has been successful and has demonstrated smooth execution of ATLAS applications on the integrated testbed• Many interested users and many sites available to join• DataManagement not fully exploited but possible (yet tested)• Possibility to enlarge pool of resources for next phases of DC1

interoperability demos and atlas applications luca vaccarossa (infn - milano)

Documents