chep 2000 smart resource management software in high energy physics wolfgang gentzsch and lothar...
TRANSCRIPT
CHEP 2000
Smart Resource Management Software
in High Energy Physics
Wolfgang Gentzsch and Lothar LippertGridware GmbH & Inc.
Padua, 9 February 2000
Technical Requirements and Features what do we offer to help HEP Computing
CHEP 2000Resource Management with CODINE / GRD
Gridware - The Company
Technology Leader in Resource Management
A special offer to the HEP community
Our answer to falling hardware-prices
Technical Requirements and Features
• Array Jobs• Advanced Queue Concept• Policy Management• Separation of Components• Solutions for mixing interactive and batch• Simplified system administration• AFS Support • CORBA Interface• All “classic” Features• Availability
Array Jobs
#!/bin/sh...
1 single Submit-Command for thousands of similar jobs
Example: qsub -t 1-1000:1 jobscript.sh
creates 1000 instances of a single job The whole array can be (also partly) manipulated (deleted, suspended, ...) with 1 command unlimited number of instances
Job
Advanced Queue Concept
The whole cluster can be adressed Soft requests are supported No empty queues while others are more than full each host can be treated with different policies users just request resources
higher efficiency
“Emergency Room Concept”
Job ClusterDispatch
Job
Q1
Q2
Example: qsub -l mem_free=10M jobscript.sh
Cluster is split Queues may run empty users have to decide for a queue Job has to stay in line also if other resources are unused
“Grocery Store Concept”
Example: qsub -q 10MQ jobscript.sh
Policy Management
FairshareOverride SystemOverride SystemBoosts temporarily project/job/group/department
Share Utilization
Time
Raise group
Execute jobs earlier
20%Group1
30%Group2
50%Group3
CODINEMASTER
FlexLM
Load/L icenseR eports
CODINESCHEDULER
W here to?W hen?
L icenses?Load?P olicy?
W eekend?P ara lle l?
C ho ice
Launch job /m an ipu la te job
de liver resu lts(exit sta tus,
accounting, ...)
L icense checkstart ca lcu la tion
in form user
Execution-Host
request licensereport
C om pute JobA rray
O utput-F iles
Hidden from the user
S ubm itM on ito rD e le te
M igra te Job...
Separation of Master and Scheduler
Scalability high performance good response time faster job placement
Separation of Components
Simplified system administration
No daemon restarts necessary
Add machines ‘on the fly’
Ability to install the entire cluster from one workstation
No submit daemons or configuration needed for client
Optimized architecture provides reliability
Conifiguration changes without any pain
What else?
CORBA InterfaceAFS Support
All “classic” Features
Interactive vs. Batch
accounting, monitoring, suspension, sensors ...
time windows automatic suspend migration, ...
Availability all leading unix platforms
The company
GENIAS Chord
based in Germany European Union funded projects R&D company
located in California leader in sales of RMS
Technology leader in Resource Management
Goal: make CODINE world standard in Resource Management
Our experience
EU funded research projects REMUS UNICORE...
Reseach & Development DESY Zeuthen (long relationship) CASPUR (recently switched to CODINE) MPI (Max Planck Institutes) ...
Industry
BMW SAAB SIEMENS ...
