a linux pc farm for physics analysis at the zeus experiment marek kowal, krzysztof wrona, tobias...

A Linux PC Farm for Physics A Linux PC Farm for Physics Analysis at the ZEUS ExperimentAnalysis at the ZEUS Experiment

Marek Kowal, Krzysztof Wrona, Tobias Haas, Ingo Martens, Rainer Mankel

DESY, Notkestrasse 85, Hanburg, Germany

http://zarah.desy.de/

Marek Kowal [email protected]

A Linux PC farm - plan of talkA Linux PC farm - plan of talk

Overall statusKey issuesHardware and SoftwareNext steps


Overall statusOverall status

First reconstruction farm working since 1997

Right now farm consists of 47 PCsBoth reconstruction and analysis

software runs efficiently on PCs


Key issuesKey issues

Big computing powerBig IO rateEasy user interfaceMaintenance Prices


Hardware - introductionHardware - introduction

“worker” PCs - 45“server” PCs - 2Fileservers - 3 TBOld SGI Multiprocessor machines -

44 processorsNetwork


PCs - commodity hardwarePCs - commodity hardware

Farm built over past three years Number of PCs processor memory IDE SCSI

– 16 B PPro 200 64MB 2GB -

– 1 S PPro 200 128MB 2GB3x8GB

– 19 B PII 350 128MB 6GB -

– 1 S PII 350 256MB 6GB 3x8GB

– 10 B PIII 450 128MB 8GB -

Each PC equipped with 100Mb network card


Fileservers - SGIFileservers - SGI

Origin 2000, 4xIP27 195MHz, 0.75GB RAM, HIPPI 800Mb, 1Gb

Challenge DM, 4xIP19 100MHz, 384 MB RAM, HIPPI 800Mb

SCSI discs - 2TBFibre Channel (!) discs - 1TB


SGI Challenge XLSGI Challenge XL

total of three machinestotal of 44 processors (IP19,IP25)

/20,16,8/4.3GB RAM /1.5,1,1.8/HIPPI 800Mb


Network hardwareNetwork hardware

zarah 1 (2 0 ) zarah 3 (8 )

zarah 2 (1 6 )

P C 1 P C 2

P C ? ? P C 4 7

C IS C O 4 8 x1 0 0 M b , 2 x1 G bS W ITC H

O rig in 2 0 0 0 2 TB C h a llen g e D M 1 TB

H IP P I G ig a R ou te r


Software - introductionSoftware - introduction

BATCH SystemJob submissiontpfs & RFIOWWW interface


Batch systemBatch system

NQS and LSF evaluated, LSF choosen for PCs

LSF– possibility to define load window– possibility to define resource

requirements for job (HDD!)SGI Challenges - still NQS


Job submission softwareJob submission software

Allows to submit jobs (binaries & data) to batch system

Each job is allocated its own working directory

Operations supported: submission, retreival, querying status, listing, killing, purging

Avaliable for: Linux, Solaris, IRIX, OSF1, Windows NT


tpfstpfs

tpfs - transparent access to data stored on robots (hard copy) and discs (cached copy)

automatic staging of files upon trial to open them


RFIORFIO

statelessavaliable as dynamically loaded

library (DLL)

> export LD_PRELOAD_PATH=librfio.so> some_program


WWW interfaceWWW interface

queues status / steeringsystems’ load and statisticsdiscs avaliability and free space

reportsstaging status and statistics


Maintenance & CostsMaintenance & Costs

More space required for PCs than for Challenges (Racks!)

No real console avaliable (BIOS problems!)

Synchronization of software - AFSPrice: 3000DM (PC, rack space,

switch port, cabling and LSF licence)


Next steps - hardwareNext steps - hardware

complete removal of SGI machines except for Origin2000 fileserver

further PCs to be addedincrease in size of discs attached to

fileserver - up to 4.5 TB till the end of 2000


Next steps - softwareNext steps - software

Development of new job submission software written as JAVA applet accesible via WWW

New version of RFIO software with support for Disc Cache project started at DESY (see poster session - Patrick Fuhrmann)


Have you got any questions?Have you got any questions?

……...

A Linux PC Farm for Physics A Linux PC Farm for Physics Analysis at the ZEUS ExperimentAnalysis at the ZEUS Experiment

Marek Kowal, Krzysztof Wrona, Tobias Haas, Ingo Martens, Rainer Mankel

DESY, Notkestrasse 85, Hanburg, Germany

http://zarah.desy.de/

a linux pc farm for physics analysis at the zeus experiment marek kowal, krzysztof wrona, tobias...

Documents