a uk computing facility

A UK Computing Facility

John Gordon

RAL

October ‘99 HEPiX Fall ‘99

Data Size

• Event Rate 109 events/year

• Storage Requirements (real & simulated data)~300TByte/year

• UK Physicists want access to data for analysis

• 2TB in 1999, 4TB/year 2000 on data and simulation


Why don’t UK Physicists use SLAC?

….and SLAC is already heavily used.


Existing UK Facilities

• Shared facilities in UK are HP, Intel/Linux and Intel/NT.

• BaBar mainly use Suns

– Historically, UK lacking in Suns in HEP departments

• BaBar have Sun E3500, 4 cpus, 2GB of memory at RAL - Bought for program development

• Several hundred GB of disk

• Plus a few desktop machines in universities


BaBar bid for more

• BaBar went to a UK Government research fund and bid for $1.8M for UK BaBar facilities

• They were awarded ~$1.2M at the start of this year for:

– A central server at RAL with several TB which will receive data from SLAC.

– Server and disk in 10 UK universities

– Co-operating databases across the UK

– One extra staff member to achieve this


Actual Equipment

• Sun vs Compaq

• Sun won.

• RAL • 5 Universities (Bristol, Edinburgh,Imperial, Liverpool, Manchester) with • 4 Universities (Birmingham, Brunel, QMW, RHBNC) with

• E4500 server -• 6x400MHz cpus, 4GB memory

• 5TB of formatted disk in 27 A1000 RAID arrays• 6 UWSCSI busses• DLT7000 stacker• 7 fast ethernet adaptors

• E250, 2 x400MHz cpus, 1GB, 3xA1000 = 0.5TB

• E450, 3x400MHzcpus, 2GB• 5x A1000, = 1TB


Setup at RAL (early experience)

• Equipment delivered and installed

• Filesystems limited to 1TB

– used 4xA1000 => 720GB striped(?)

• 5.5 Million events brought from SLAC

• E3500 acts as a front-end, E4500 holds data, both runs batch jobs

• E4500 also AMS server to other systems.

• LSF cluster on 2 Suns.

• Who else is running large data on Suns?


OOSS

• Andy Hanushevsky visited in September and installed his OOFS and OOSS

• This provides a layer which interfaces Objectivity to the Atlas Datastore (cf HPSS at SLAC)

• All the disk space runs under the control of OOS which acts as a cache manager

• Current level of Objectivity/AMS doesn’t allow OOS to retrieve data transparently from the robot but data can be easily brought on-line by prestaging


Network Plans

• A single server in a university on fast ethernet can suck data from RAL at rates which will be unpopular with other sharing the institutes connections to the WAN.

• Pilot to establish tunnels over JANET using spare ATM capacity


Manchester

RAL IC

2MB2MB


Purpose of Trial

• Since bandwidth will be small the trial will not necessarily give better throughput

• Establish whether end-to-end connection over PVCs works

• Establish whether the different management domains can reach a common, working solution

• Check that the routing works

• Should be simple to increase bandwidth later


Original Data Model

• Data to RAL by tape

• Model I - all TAG data at other sites; pull detailed data from RAL

• Model II - frequently-accessed events stored in full at other sites; replication from RAL

• Investigate methods of copying, updating, replicating databases over WAN


New Data Model(?)

• BaBar currently has performance limitations

• Working on non-Objectivity solution NOTMA

• DST using ROOT i/o

• 10**9 events = 280GB

• Likely that universities will want all events locally

• Detailed events stored at RAL in Objectivity


Conclusion

• RAL moving in opposite direction from HEPCCC proposal - more flavours of unix on new hardware platform. BaBar will be using Linux soon for simulation though

• A bigger scale of disk data handling for one experiment.

• Data synchronisation over WAN

– (SLAC-RAL-UK Universities)

a uk computing facility

Documents