fokke dijkstra, bob dröge high performance computing and … · 2015-10-02 · 24-9-2012 using the...
TRANSCRIPT
24-9-2012
Using the Millipede cluster - IFokke Dijkstra, Bob Dröge
High Performance Computing and Visualisation group
Donald Smits Centre for Information Technology
24-9-2012
General introduction
› Course aimed at beginners
• No knowledge about clusters or Linux necessary
• Examples based on UWP Windows installation
• Some pointers for Linux or Mac OS X users
› Topics – Part I
• Connecting to the cluster
• Using the Linux command line
• Editing files
24-9-2012
› How do I get onto the system?
24-9-2012
Millipede cluster
userfront-end node
internet fast network
Cluster nodes
CIT
Office somewhere
24-9-2012
SSH
› SSH protocol used to connect to the cluster
• Standard interface for Unix systems
• Encrypted network traffic
› Software for Windows called PuTTY:• http://www.chiark.greenend.org.uk/~sgtatham/putty/
• Open source software, freely available
• Already installed on UWP
› Use terminal window under Linux or Mac OS X
› Normally university account id used (p- or s-number)
› Hostname millipede.service.rug.nl
24-9-2012
PuTTY start window
24-9-2012
PuTTY host key check
• This happens when you connect for the first time
• Identity of system not known by PuTTY client
• May also happen after an update on the login node
24-9-2012
PuTTY login
24-9-2012
› Help, where is my data?
24-9-2012
File organisation
› In windows directory structure separated by \
› Drive letter included
C:\Users\Peter\My Documents\testfile.txt
› In Linux directories separated by /
› Drive letters not used, everything appears as a directory name/home/peter/My Documents/testfile.txt
› Note that the use of “spaces” is difficult on the command line
› Use quotes if you really need to use them
”/home/peter/My Documents/testfile.txt”
› Case-sensitive: Myfile.txt ≠ MyFile.txt
24-9-2012
A Linux system concept: the home directory
› A unique home directory for each user.
› Every user has one.
› The ssh session (putty terminal) starts in this directory.
› On millipede looks like:/home/username, e.g./home/p742111
24-9-2012
File transfer clients
› Windows
• WinSCP is a file transfer program for Windows
• http://winscp.net
• Open source, freely available
• Not pre-installed in UWP
› Linux
• Command line scp or sftp
• Nautilus file manager, Konqueror (fish://), FileZilla
› Mac OS X (not tested)
• Command line scp or sftp
• CyberDuck (Open source), FileZilla (Open source), Fetch (Free academic license)
24-9-2012
Copying files: WinSCP
24-9-2012
WinSCP host key check
24-9-2012
WinSCP file operations: copy, rename, move, delete
Local filesMillipede
24-9-2012
› But it runs Linux! Introducing the Linux command line
24-9-2012
Basic commands: ls
› ls lists files:$ ls
Job1_script.sh Task1.in.mat Task1.state.mat Task2.jobout.mat
matlab_metadata.mat Task1.jobout.mat Task2.common.mat Task2.out.mat
Task1.common.mat Task1.out.mat Task2.in.mat Task2.state.mat
$ ls -l
total 32
-rw-rw-r-- 1 peter peter 1354 Jan 11 10:10 Job1_script.sh
-rw-rw-r-- 1 peter peter 587 Jan 11 10:10 matlab_metadata.mat
-rw-rw-r-- 1 peter peter 0 Jan 11 10:10 Task1.common.mat
-rw-rw-r-- 1 peter peter 690 Jan 11 10:10 Task1.in.mat
-rw-rw-r-- 1 peter peter 0 Jan 11 10:10 Task1.jobout.mat
-rw-rw-r-- 1 peter peter 889 Jan 11 10:10 Task1.out.mat
-rw-rw-r-- 1 peter peter 7 Jan 12 09:47 Task1.state.mat
-rw-rw-r-- 1 peter peter 0 Jan 11 10:10 Task2.common.mat
-rw-rw-r-- 1 peter peter 690 Jan 11 10:10 Task2.in.mat
-rw-rw-r-- 1 peter peter 0 Jan 11 10:10 Task2.jobout.mat
-rw-rw-r-- 1 peter peter 889 Jan 11 10:10 Task2.out.mat
-rw-rw-r-- 1 peter peter 7 Jan 12 09:47 Task2.state.mat
24-9-2012
Basic commands: cd
› cd changes to a directory$ ls
Readme.txt Job1 Job2 Job3
$ cd Job1
$ ls
Job1_script.sh Task1.in.mat Task1.state.mat Task2.jobout.mat
matlab_metadata.mat Task1.jobout.mat Task2.common.mat Task2.out.mat
Task1.common.mat Task1.out.mat Task2.in.mat Task2.state.mat
› “..” points to directory above
$ cd ..
$ ls
Readme.txt Job1 Job2 Job3
› “.” points to current directory$ ./myprog
$ cp ../AnotherProgram .
24-9-2012
Basic commands: mkdir/rmdir
› mkdir creates a directory:$ ls
Job1_script.sh Task1.in.mat
$ mkdir testdir
$ ls
Job1_script.sh Task1.in.mat testdir
› rmdir removes a directory (must be empty):
$ rmdir testdir
$ ls
Job1_script.sh Task1.in.mat
24-9-2012
Basic commands: cp
› cp copies files (and/or directories):$ ls
Job1_script.sh Task1.in.mat Task1.state.mat Task2.jobout.mat
matlab_metadata.mat Task1.jobout.mat Task2.common.mat Task2.out.mat
Task1.common.mat Task1.out.mat Task2.in.mat Task2.state.mat
› Copy single file$ cp Task1.state.mat Task3.state.mat
$ ls
Job1_script.sh Task1.in.mat Task1.state.mat Task2.jobout.mat
matlab_metadata.mat Task1.jobout.mat Task2.common.mat Task2.out.mat
Task1.common.mat Task1.out.mat Task2.in.mat Task2.state.mat
Task3.state.mat
› Copy multiple files$ cp *.mat ../newdirectory
$ cp -r mydir newdir
24-9-2012
Basic commands: rm
› rm removes files (and/or directories):$ ls
Job1_script.sh Task1.in.mat Task1.state.mat Task2.jobout.mat
matlab_metadata.mat Task1.jobout.mat Task2.common.mat Task2.out.mat
Task1.common.mat Task1.out.mat Task2.in.mat Task2.state.mat
› Remove single file$ rm Task1.state.mat
› Remove multiple files$ rm *.mat
$ rm Task?.common.mat
$ rm *
everything in current directory is removed. Make sure you know your current directory! To avoid this go a directory up, then remove the desired directory
$ cd ..
$ rm dirname/*
› Remove a directory with all files and subdirectories$ rm -r mydir
24-9-2012
Basic commands: mv
› mv moves/renames files:$ ls
Job1_script.sh Task1.in.mat Task1.state.mat Task2.jobout.mat
$ mv Job1_script.sh jobscript.sh
$ ls
jobscript.sh Task1.in.mat Task1.state.mat Task2.jobout.mat
› Can move a whole directory to another place
$ mv Job1Directory ../jobDirectory
24-9-2012
Useful tricks
› <Tab> does auto completion of file names$ ls
Job1_script.sh Task1.in.mat Task1.state.mat Task2.jobout.mat
$ mv J<Tab>
results in
$ mv Job1_script.sh
› Arrows keys ( ) can be used to scroll through list of previous commands
› Ctrl-A or Home takes your cursor to the beginning of the commandline
› Ctrl-E or End takes your cursor to the end of the command line
› Ctrl-R enables search through previous commands.
› Ctrl-C interrupts the current program
› Ctrl-D logs you out (ends the session)
24-9-2012
Other useful commands
› pwd: prints current directory$ pwd
/home/peter/testdir
› echo: puts some text on the screen$ echo Hello world!
Hello world!
› less: Look at the contents of a file (exit using q)$ less my_output_file
› man: Get information about a command (exit using q)$ man ls
› copy-paste: select the piece of text of interest
• Ctrl+Insert – copy the text
• Shift+Insert - paste the copied text
24-9-2012
Environment variables
› Linux scripts make extensive use of environment variables
› Prefixed by $
› Expanded when the command is run
› Examples
$ cd $HOME
Will change into home directory $HOME is replaced with e.g. /home/f204524
$ echo $USER
Will display the user name $USER is replaced by e.g f204524
› If ambiguity can arise use ${VARIABLE}
24-9-2012
Redirecting in- and output
› Input and/or output of programs can be redirected to another program or to a file
› Send output to file using “>”$ ls -l > files.txt
› Read input from a file using “<“$ myprog < inputfile > outputfile
› Pipe output from a command into another command$ qstat | less
› Append output to a file usign “>>”$ ls –l >> files.txt
24-9-2012
› Do I really have to know what ctrl-x ctrl-s or :wq does? Editing files
24-9-2012
Editing files through WinSCP
24-9-2012
Nano editor
$ nano testjob
24-9-2012
Linux and windows text
› Windows uses two characters (carriage return and linefeed) to end a line
› Linux and Mac OSX only use the linefeed character to end a line
› Older Macs only use the carriage return character instead
› This may sometimes give you problems
› Tools dos2unix & unix2dos & mac2unix can convert text if necessary
Full Linux course available
Topics:› History and philosophy behind Linux› Installing and configuring Ubuntu, installing software› The desktop environment› Controlling processes › The Linux filesystem and the commands available› Using bash (the Bourne-Again Shell)› Connecting programs together› Basic scripting
http://www.rug.nl/cit/onderwijs/cursus/linux
24-9-2012
Using the Millipede cluster - IIFokke Dijkstra, Bob Dröge, Ana Oprescu
High Performance Computing and Visualisation group
Donald Smits Centre for Information Technology
24-9-2012
General introduction
› Course aimed at beginners
• This part assumes knowledge about the Linux command line, file transfers and editing files
› Topics – Part II
• What is a cluster
• Cluster storage
• Module environment
• Submitting jobs
24-9-2012
HPC/V
› High performance computing and visualisation group
› HPC Facilities
• Millipede cluster
• IBM BlueGene/P
• Grid cluster
› Visualisation
• Cave
• Theatre
• Scientific data
• Virtual reality
24-9-2012
› What is a computer cluster?
24-9-2012
Computer cluster
› A cluster is a collection of computers connected by a network
› A single front-end
› Lots of computers in the background for running tasks
› 1994 first cluster of commodity PCs at NASA
› Millipede cluster looks quite different
› Most clusters run the Linux operating system
Cluster at Michigan Technical University
24-9-2012
Cluster in more detail
userfront-end node
internet fast network
Cluster nodes
CIT
Office somewhere
24-9-2012
How does it work?
internet
sshscp
24-9-2012
Why a command line?
› Easier over the network
› Scripts are easier to run many times
› Instead of:
› One can use a small program
24-9-2012
SSH
› SSH protocol used to connect to the cluster
• Standard interface for Unix systems
• Encrypted network traffic
› Software for Windows called PuTTY:• http://www.chiark.greenend.org.uk/~sgtatham/putty/
• Open source software, freely available
• Already installed on UWP
› Use terminal window under Linux or Mac OS X
› Normally university account id used (p- or s-number)› Hostname millipede.service.rug.nl
24-9-2012
Millipede overview
› 4 node types
• 1 login node
• 12 cores, 24 GB memory
• 231 standard nodes
• 12 cores, 24 GB memory
• 16 large nodes
• 24 cores, 128 GB memory
• 1 special SMP node
• 64 cores, 478 GB memory
› Network
• 20 Gbit/s Infiniband network
› 3 File systems
Standard modern desktop PC
• 4 cores
• 8 GB memory
24-9-2012
Accessing storage on UWP
› Cluster storage /home and /data can be reached on UWP
• Documentation available• http://www.rug.nl/cit/hpcv/publications/docs/millipede_man/cl
uster-UWP.pdf
• Example: Start -> Run
24-9-2012
File transfer clients
› Windows
• WinSCP is a file transfer program for Windows
• http://winscp.net
• Open source, freely available
› Linux
• Command line scp or sftp
• Nautilus file manager, Konqueror (fish://), FileZilla
› Mac OS X (not tested)
• Command line scp or sftp
• CyberDuck (Open source), FileZilla (Open source), Fetch (Free academic license)
24-9-2012
/home file system
› Each user has a directory within /home
• E.g /home/f205229
› 2 * 7 TB in total (mirrored)
› Backed up
› Default quota of 10 GB per user
• Prevents system from uncontrolled filling up
• Contact us if you need more space
• Use “quota” command to see the status
› Shared over NFS to all nodes
• Limited performance
24-9-2012
/data & /data/scratch
› 110 TB file system based on Fraunhofer Parallel File System (FhGFS)
› Faster than /home for very large files
› Currently slow for small file access
› /data
• No active quota, please limit yourself to 200 GB
• Contact us if you need more space
› /data/scratch
• Temporary storage
• May be cleaned by system administrators
24-9-2012
/local
› Local disk on all of the nodes
› Only reachable on that specific node
› Needs to be cleaned up within the job at the end
› $TMPDIR points to a directory that is created per job on the master node of the job
• Automatically removed after the job has finished
24-9-2012
› What can it do for me?
24-9-2012
For whom?
› Account available for University staff & students
• Undergraduate students through supervisor/teacher
• Provide contact details and short description of planned use
› Access through ssh protocol
• millipede.service.rug.nl
24-9-2012
What can it be used for?
› Applications must be able to run under Linux
• Compile the application yourself
• Preinstalled applications
• matlab, R, gromacs, …
• Run anywhere languages
• Java, Python, Perl, ….
• Some programs can be run under a windows emulator
• License must be available
24-9-2012
No direct user interaction
› Applications must run without user intervention
• You don’t know when the application will start
• Makes running multiple times difficult
› No graphical user interface
• If no user interaction is necessary the output can be sent to a dummy display
• Use case: saving matlab and R plots
› Input and output through files
Licenses
› If you want to use commercial software a license may be necessary
› Provided by the university
• May be installed system wide
• Problem with Linux versions or number of licenses
› Provided by yourself
• Install software in home directory
› Other cases may be more difficult
24-9-2012
24-9-2012
Use cases
› Application areas that can be used:• Data analysis using Matlab or R• Molecular dynamics using Gromacs• fMRI image analysis• Quantum chemistry• High energy physics using root, geant4• Run your own software in Fortran, C, C++
› General scenarios:• Long running calculations• Parallel calculations• Many calculations
24-9-2012
› Where is my (insert favourite software package)?Available software, using the module system
24-9-2012
Modules environment
› Software organised through modules system
› Environment set for software when module is loaded
› Useful commands:
• module avail
• module list
• module add
• module remove
• module initadd
24-9-2012
Module examples
$ module avail
---------------------------- /cm/local/modulefiles -----
cluster-tools/5.0 freeipmi/0.7.11 shared
cmd module-info use.own
dot null version
---------------------------- /cm/shared/modulefiles ----
R/2.10.1 hdf5/1.8.5
R-tcltk/2.10.1 hpl/2.0
acml/gcc/64/4.3.0 intel/compiler/32/11.
acml/gcc/mp/64/4.3.0 intel/compiler/64/11.
....
$ icc
-bash: icc: command not found
$ module add intel/compiler/64
$ icc --version
icc (ICC) 11.1 20090630
Copyright (C) 1985-2009 Intel Corporation. All rights reserved.
$ module remove intel
24-9-2012
Useful software
› Compilers for Fortran, C and C++
• GNU compilers
• Intel compilers
› Other languages
• Java, Python, Perl
› MPI libraries
• OpenMPI, MVAPICH2
› Scheduler software – torque, maui
› Matlab and R
24-9-2012
› But I only see a single computer!
24-9-2012
Login node
› Front-end node
› Used for access to the cluster
• Login
• Data transfers
• Job submission
• Editing & Compiling programs
• (Very) small tests
• Hardware same as standard nodes
24-9-2012
Scheduling system (1)
› Users write job descriptions
› Scheduler takes job description and finds matching resource
› Scheduler tries to make optimal use of the resources
› If no resources are available, jobs have to wait in a queue
› Priority in the queue determined by usage of system in the recent past
24-9-2012
≤ 1 day ≤ 3 days ≤ 10 days
Standardnodes
nodes nodesmedium nodeslong
Big nodes(“quads”)
quads quadsmedium quadslong
Special SMP node
smp smpmedium smplong
≤ 30 minutes (only for testing!)
Standard nodes
short
Scheduling system (2)
24-9-2012
Multiple jobs per node
› Nodes have at least 12 cores
› Scheduling full nodes to users that only use a single core is a waste of resources
› Users can still request a complete node if necessary
› Jobs on a single node will influence each other
• Disk space (space and bandwidth)
• Memory (space and bandwidth)
• Can sometimes be problematic
24-9-2012
Node types
› 231 12 core nodes
• 2 x 6 core AMD Opteron @ 2.6 GHz
• 24 GB memory
• Bulk of the cluster capacity
› 16 16 core nodes
• 4 x 6 core AMD Opteron @ 2.6 GHz
› Reachable through scheduler
› Default multiple jobs per node!
Infiniband switch
24-9-2012
SMP node
› Single 64 core node
• Combination of 8 nodes into a SMP node using ScaleMP
• 8 * 2 * 4 core Intel Xeon E5520 “Nehalem” @ 2.3 GHz
• 4 * 48 + 4 * 96 GB memory = 576 GB (- ~ 90 GB used for ScaleMP, including vSMP cache)
› Reachable through scheduler
› Large shared memory jobs
• Large memory
• More cores
› Multiple jobs on the node
24-9-2012
› What is a job script?
24-9-2012
Job script
› Job script tells the system what you want to do
• List of steps, small program
› Includes requirements needed to be able to run it
• Amount of memory necessary
• How many cores or nodes you need
• Time you will need before it is finished
24-9-2012
Job requirements
› Can be put in job script using#PBS
› Requirements specified using -l option:
• Wall clock time
• #PBS -l walltime=hh:mm:ss
• Number of nodes and cores
• #PBS -l nodes=n:ppn=m
› Specific node types using -q option:
• #PBS -q quads
24-9-2012
Memory requirements
› Memory requirements can be specified using#PBS -l mem=nGBn is the amount of memory you need
› If you need a lot of memory and a few cores• Consider using the quad nodes• Consider requesting the number of cores
corresponding to the amount of memory› 2 GB / core on standard nodes› 5.3 GB /core on quad nodes› Currently not everyone gives a memory requirement
• Scheduling may be wrong
24-9-2012
Job properties
› Also using #PBS lines or on the command line
› Name of the job
• #PBS -N name
› Combine standard output and error into a single file
• #PBS -j option
• oe : Merge into standard output file
• eo: Merge into standard error file
› Sending e-mails
• #PBS -m options
• Options is a combination of n (no), b (begin), e (end) and/or a (aborted)
• Mail can be problematic in case of Millipede problems
24-9-2012
Job script itself
› Contains Linux commands
• cd, mkdir, etc.
› Sample script:
#PBS -l nodes=1:ppn=2#PBS -l walltime=00:30:00#PBS -N testjob
cd $PBS_O_WORKDIR./myprog
24-9-2012
Useful environment variables
$HOME: your home directory
$USER: your user name
$TMPDIR: Temporary directory created for your job. Removed after your job has finished
$PBS_O_WORKDIR: Directory from which the job was submitted
$PBS_NODEFILE: Points to a file with a list of the allocated nodes (one entry for each core on the node)
$PBS_JOBID: Id of job, useful for creating unique files or directories for a job
24-9-2012
Submitting jobs
› qsub jobscript
$ qsub testjob1870565.millipede.cm.cluster
Job id
24-9-2012
Checking job status (1)
› qstat
$ qstat Job id Name User Time Use S Queue------------------------- ---------------- --------------- -------- - -----1820058.millipede Dimeric-4 p552330 21054:10 R nodeslong 1820631.millipede SM16-DISORDER p552330 5204:21: R nodeslong 1824996.millipede Complex p982535 26127:01 R quadslong 1832677.millipede DEMIX p834345 2710:47: R nodeslong 1835800.millipede BIG-DUPC-3 p552330 18479:20 R nodeslong 1835801.millipede BIG-DUPC-4 p552330 17856:18 R nodeslong 1835802.millipede BIG-DUPC-5 p552330 17798:52 R nodeslong 1843168.millipede aaTHLmem_45 s8656343 1438:06: R nodeslong 1843186.millipede walp-dupc-nochol p539241 2729:18: R nodeslong 1846698.millipede DEMIXTC p834345 2525:40: R nodeslong 1846798.millipede RAFT-DIGAL p834345 2516:58: R nodeslong 1847258.millipede testjob peter 0 Q nodes 1847358.millipede 120cpus p523301 11484:12 R nodeslong 1849156.millipede genome p735434 93:30:29 R nodeslong 1849216.millipede myjob p724263 105:51:1 R nodeslong 1849376.millipede 120cpus p523301 11037:28 R nodeslong 1849433.millipede PMF0 p633535 1103:37: R nodeslong 1849434.millipede PMF1 p633535 1103:29: R nodeslong .........
24-9-2012
Checking job status (2)
$ qstat -u peter
millipede.cm.cluster: Req'd Req'd Elap
Job ID Username Queue Jobname SessID NDS TSK Memory Time S Time-------------------- -------- -------- ---------------- ------ ----- --- ------ ----- - -----1847258.milliped peter nodes testjob -- 1 1 -- 01:00 Q --
Status:
Q: queued (waiting)
R: running
E: finishing
H: halted (waiting for another job to finish)
24-9-2012
Checking job status (3)
› showq (can also use -u option)$ showqACTIVE JOBS--------------------JOBNAME USERNAME STATE PROC REMAINING STARTTIME
1869946 dijkstra Running 2 00:02:12 Mon Jun 27 13:10:521869948 dijkstra Running 2 00:04:00 Mon Jun 27 13:12:401869949 dijkstra Running 2 00:05:18 Mon Jun 27 13:13:581869951 dijkstra Running 2 00:05:18 Mon Jun 27 13:13:581869952 dijkstra Running 2 00:05:18 Mon Jun 27 13:13:581869953 dijkstra Running 2 00:05:18 Mon Jun 27 13:13:581869969 dijkstra Running 2 00:07:23 Mon Jun 27 13:16:031869970 dijkstra Running 2 00:07:23 Mon Jun 27 13:16:031870063 dijkstra Running 2 00:53:54 Mon Jun 27 14:02:341870064 dijkstra Running 2 00:53:54 Mon Jun 27 14:02:341870065 dijkstra Running 2 00:53:54 Mon Jun 27 14:02:341870066 dijkstra Running 2 00:54:28 Mon Jun 27 14:03:081870361 dijkstra Running 2 1:12:54 Mon Jun 27 14:21:341870362 dijkstra Running 2 1:13:36 Mon Jun 27 14:22:161870363 dijkstra Running 2 1:13:36 Mon Jun 27 14:22:161870386 dijkstra Running 2 1:13:36 Mon Jun 27 14:22:161870406 dijkstra Running 2 1:16:36 Mon Jun 27 14:25:16......
24-9-2012
Checking job status (4)
...
...1867510 p539241 Running 24 9:16:45:00 Mon Jun 27 07:54:321867515 p539241 Running 24 9:16:45:21 Mon Jun 27 07:54:531869709 s7104947 Running 12 9:19:46:57 Mon Jun 27 10:56:29
281 Active Jobs 2748 of 3208 Processors Active (85.66%)224 of 247 Nodes Active (90.69%)
IDLE JOBS----------------------JOBNAME USERNAME STATE PROC WCLIMIT QUEUETIME
1850797 p552330 Idle 120 10:00:00:00 Fri Jun 24 16:58:271868042 p552330 Idle 24 10:00:00:00 Mon Jun 27 08:41:181868046 p552330 Idle 24 10:00:00:00 Mon Jun 27 08:41:311868049 p552330 Idle 24 10:00:00:00 Mon Jun 27 08:41:431868052 p552330 Idle 24 10:00:00:00 Mon Jun 27 08:41:541868055 p552330 Idle 24 10:00:00:00 Mon Jun 27 08:42:051868059 p552330 Idle 24 10:00:00:00 Mon Jun 27 08:42:211868063 p552330 Idle 24 10:00:00:00 Mon Jun 27 08:42:341868066 p552330 Idle 24 10:00:00:00 Mon Jun 27 08:42:451868069 p552330 Idle 24 10:00:00:00 Mon Jun 27 08:42:551868072 p552330 Idle 24 10:00:00:00 Mon Jun 27 08:43:061868074 p552330 Idle 24 10:00:00:00 Mon Jun 27 08:43:161868078 p552330 Idle 24 10:00:00:00 Mon Jun 27 08:43:27......
24-9-2012
Checking the results
› If the job disappears from qstat it has finished
› Output files written to same directory as from which the job was submitted
› jobname.ojobnum and jobname.ejobnum filese.g. testjob.o1873505 and testjob.e1873505
› .o file contains everything written to standard output
› .e file contains everything written to standard error
24-9-2012
Oops, I did not want to run that!
› qdel
$ qsub testjob1870679.millipede.cm.cluster$ qstat -u peter
millipede.cm.cluster: Req'd Req'd Elap
Job ID Username Queue Jobname SessID NDS TSK Memory Time S Time-------------------- -------- -------- ---------------- ------ ----- --- ------ ----- - -----1847258.milliped peter nodes testjob -- 1 1 -- 01:00 Q --1870679.milliped peter short test -- 1 4 500mb 00:20 Q --$ qdel 1870679$
24-9-2012
A real example
› A job script that runs R code:
#PBS -N R_job#PBS -l walltime=00:01:00#PBS -l nodes=1:ppn=1#PBS -l mem=1GB#PBS -q short
cd $PBS_O_WORKDIRmodule load R/2.15.0Rscript myscript.r
24-9-2012
Another real example
› A job script that runs Matlab code:
#PBS -N matlab_job#PBS –l nodes=1:ppn=1#PBS –l walltime=00:00:30#PBS -j oe#PBS –q short
cd $PBS_O_WORKDIRmodule load matlabmatlab -nodisplay -r mycode
Code in file mycode.m
24-9-2012
Remarks on Matlab
› Matlab needs a license to be able to run
› Number of concurrent users is limited
› Matlab code can be compiled using Matlab compiler
• Running the compiled code does not require a license
› Parallel computing toolbox supported
• Can use up to 8 cores
• Parallel Matlab code cannot be compiled
24-9-2012
Graphical output
24-9-2012
Running applications with a GUI
› Only on the login node!
› Needs fast network connection
• May work very slow at home
› Linux uses X protocol for displaying graphics
› X graphics can be shown on another display
› Needs X server software on your desktop
• Xming (Free version available), easier to usehttp://www.straightrunning.com/XmingNotes/
• X-deep/32 (UWP)
24-9-2012
X-Deep/32 (1)
› Start: X-Deep/32 X-server
off
24-9-2012
X-Deep/32 (2)
Select X-Deep/32 icon
options
24-9-2012
X-Deep/32 (3)
off
24-9-2012
Xming
Keep blocking since we will be using
connections through ssh
24-9-2012
PuTTY X connections
• Only necessary when you want to display programs with a graphical user interface on your local desktop
• You will need an X server for that
Example: Matlab
› To start Matlab on the login node:module load matlabmatlab
24-9-2012
› Can only be used for simple tests, preparing jobs, viewing output, etcetera
24-9-2012
› Help!
24-9-2012
Support
› Support through CIT central service desk
• Phone: 3232
• E-mail: [email protected]
› Documentation online• http://www.rug.nl/cit/hpcv/publications/docs/millipede
› Comments and questions are always welcome
24-9-2012
› I want to know more!
24-9-2012
Useful documents
› Introduction to Linux by Machteld Garrels: http://tldp.org/LDP/intro-linux/html/index.html
› Bash shell guide by Machteld Garrels:http://tldp.org/LDP/Bash-Beginners-Guide/html/index.html
› Linux guide: http://www.tuxfiles.org
› Linux command line: http://linuxcommand.org/
24-9-2012
Questions?