UTA Site ReportJae Yu

UTA Site Report

3rd DOSAR WorkshopUniversity of Oklahoma

Sept. 21 – 22, 2006

Jae YuUniv. of Texas, Arlington

9/21/2006 2UTA Site ReportJae Yu

• UTA’s conversion to ATLAS experiment is in its final throes– Kaushik De is co-leading Panda development

• Part of ATLAS SW Tier 2 is at UTA• Its phase I implementation in progress

– Jae Yu focus on MonALISA based Panda monitoring– No significant DØ production work done during the past

several months• HEP group working with other discipline in shared

use of existing computing resources, notably DPCC

Introduction

9/21/2006 3UTA Site ReportJae Yu

• UTA HEP-CSE + UTSW Medical joint project• NSF MRI supported• Hardware Capacity

– Linux system• 197 CPU’s of mixture of P4 Xeon 2.4 – 2.6 GHz• Total disk: 76.2TB• Total Memory: 1GB/CPU• Network bandwidth: 68Gb/sec• Additional equipment will be purchased (about 2 more racks)

– 3 IBM PS157 Series Shared Memory system • 8 1.5GHz Power5 processors• 32 GB RAM• 6 140GB Internal Disk drives• 1 2Gb fibre Channel Adapter• 2 Gigabit Ethernet Nics

UTA DPCC

9/21/2006 4UTA Site ReportJae Yu

• Participated strongly on DØ and ATLAS MC production as well as DØ data reprocessing– Other disciplines also use this facility

• Biology, Geology, UTSW medical, etc

• Converted over for more focused ATLAS tasks• Will use opportunistic computing tactics for DØ and

other Tasks• Old farm for D0 has been shutdown and taken apart

to bunches of test clusters

UTA DPCC

9/21/2006 5UTA Site ReportJae Yu

UTA – RAC (DPCC)•100 P4 Xeon 2.6GHz CPU = 260 GHz•64TB of IDE RAID + 4TB internal•NFS File system

•84 P4 Xeon 2.4GHz CPU = 202 GHz•5TB of FBC + 3.2TB IDE Internal•GFS File system

•Total CPU: 462 GHz•Total disk: 76.2TB•Total Memory: 168Gbyte•Network bandwidth: 68Gb/sec

•HEP – CSE Joint Project•DØ+ATLAS•CSE Research

9/21/2006 6UTA Site ReportJae Yu

• Joint effort between UTA, OU, LU and UNM• Phase I completed and is up and running

– Ready for DA job receptions• Equipment for Phase II to be located in our new

CPB room being looked into

SWT2

9/21/2006 7UTA Site ReportJae Yu

SWT2 Phase II

9/21/2006 8UTA Site ReportJae Yu

• Network– Had DS3 (44.7MBits/sec) till late 2004– Increased to OC3 (155 MBits/s) early 2005– OC12 as of early 2006 – Expected to be connected to NLR (10GB/s) through

LEARN soon (http://www.tx-learn.org/) • $9.8M ($7.3M for optical fiber network) state of Texas funds

approved in Sept. 2004

Networks

9/21/2006 9UTA Site ReportJae Yu

MidlandOdessa

Lubbock

Amarillo

El Paso

Denton

Fort Worth Dallas

LongviewTyler

NacogdochesWaco

Austin

San Marcos

San Antonio

Laredo

EdinburgHarlingen

Brownsville

KingsvilleCorpus Christi

Houston

Galveston

Beaumont

College Station

Waller

Production FiberTesting Fiber

Characterized awaiting EquipmentContracted – NOT characterized

NO Contract

LEARN Fiber IRU(solid line)

LEARN Leased Lambda or Service(dashed line)

WichitaFalls

Status

9/21/2006 10UTA Site ReportJae Yu

9/21/2006 11UTA Site ReportJae Yu

Software Development Activities • MonALISA based ATLAS distributed analysis

monitoring – The feasibility has been investigated– Scalability has been tested within the UTA domain

9/21/2006 12UTA Site ReportJae Yu

Results of Scalability Test in SI95Studying the impact of the number of ApMon processes on the CPU utilization of the MonALISA service and repository

-5

0

5

10

15

20

25

30

35

40

0 10 20 30 40 50

number of ApMon processes

CP

U u

tili

za

tio

n (

SI9

5)

MonALISA

Repository

ApMon

Machine w/ ApMons

Machine w/ Repository

Machine w/ MN server

2kHz

9/21/2006 13UTA Site ReportJae Yu

Software Development Activities • MonALISA based ATLAS distributed analysis

monitoring – The feasibility has been investigated– Scalability has been tested within the UTA domain

• Hired a software specialist to focus on development of the distributed analysis system– Located at BNL– Work closely with Panda team as an integral part– Completed the code modification for apmon to Panda

9/21/2006 14UTA Site ReportJae Yu

Proposed MonALISA Based Panda Monitoring System

Client Side (Pilot, job, Scheduler)

Logging Mechanism

(File, Http)

MonALISA Logging Mechanism (ApMons)

MonALISA ServiceMonALISA Repository

Web Service client

DashBoard DB

DashBoard

or

9/21/2006 15UTA Site ReportJae Yu

ATLAS DA Dashboard• LCG sites report to One MonALISA service and one

repository– CERN colleagues implemented an ATLAS DA dashboard

• OSG Sites different– Extremely democratic Each site has its own MonALISA

server and repository– An Apmon developed for each job to report to MonALISA

server– MonALISA server will respond when prompted by the

Dashboard• Code for ATLAS OSG sites completed and undergoing

a test before release and deployment

9/21/2006 16UTA Site ReportJae Yu

CSE Student Exchange Program • Joint effort between HEP and CSE

– David Levine is the primary contact at CSE• A total of 10 CSE MS Students each have worked in SAM-

Grid team– Five generations of the student– This program ended as of Aug. 31, 2006

• New program with BNL being implemented– First set of two students started working summer 2006– Participating in ATLAS Panda projects– Will write theses for documentation

9/21/2006 17UTA Site ReportJae Yu

Conclusions• UTA’s transition from DØ to ATLAS is in its final throes • ATLAS DA work with Panda team moving along well

– Our proposal for MonALISA based DA monitoring adopted as the initial direction for ATLAS monitoring system

• Actively participating in ATLAS CSC analyses• The new network capacity of 10GB/s moving along