linux cluster and distributed resource manager
DESCRIPTION
Linux Cluster and Distributed Resource Manager using Shell ScriptTRANSCRIPT
Linux Cluster and Distributed Resource
Manager
Center for Genome Science, NIH, KCDCHong Chang Bum
Linux Cluster
• 3 Linux Cluster Machine
• KHAN
• KGENE
• BIO
KHAN
• Total 94 Nodes (1 Master + 94 work Node)
• 64Bit PowerPC 720 4-way, 16GB RAM
• 64Bit PowerPC 770 2-way, 2GB RAM
• User Space: /home1 1.6TB
• Scratch Space: /home2/scratch 935GB
• Software: /home1/biosoftware
• EIGNSTRAT, merlin, phase, plink, R ...
DRM
• Distribute Resource Manager
• Job Scheduler
• IBM LoadLeveler, PBS, OpenPBS, Torqueue, Grid Engine, Sun Grid Engine(SGE), LSF
• Job Distribute, Job Status, Queue status, Node Status, User Accounting
1,000 Serial Job
• 1,000 Data Sets
• input.1, input.2... input.1000
• program -i input.(1~1000) -o output.(1~1000)
• 1,000 Job Submit Scripts
• 1,000 times Queue Submit Commands
Job Array - LoadLeveler
• Large Data sets: 1,000(input.1 ~ input.1000)
1,000 times??
Using Script
1,000 Command Script
1,000 Job Submit Command
1,000 Command Script
1,000 Job Submit Command
Job Status
Using R Scripts• Interactive Program
• Need to use R to analyze your data
Convert Bash Shell Script??
--quiet, -q Don’t print startup message--no-save Don’t save it
Using I/O redirection Form
R and LoadLeveler