Using theEaStCHEM Research
Computing Facility
An Introduction to Linux, SLURM, CYGWINand many other cryptic acronyms
Herbert Früchtl
OverviewThe EaStCHEM Research Computing FacilityConnect to a Linux computer from Windows using
CYGWIN Installation UseIntroduction to Linux Basic Linux commands Editing with Emacs and vi Transfering files between desktop and Linux serverRunning calculations through SLURM or SGE A typical job script How to submit and monitor jobsRules of useSecurity and privacy
Current St Andrews Clusters
wardlawwith Maths & Astronomy
290 x 12-core150 TB disk space
Accessible to allSt Andrews researchers
obelix
7 x 12 + 1 x 16 cores18 TB disk space
computationalchemistry group
knox
18 x 8 cores3 TB disk space
Bühl group andcollaborators
chpc-hf63-3
6 x 8 cores2 TB disk space
Früchtl group andcollaborators
kennedywith Astronomy
54 x 32-core250 TB disk space
Accessible to allSt Andrews researchersPaid-for access possible
The St Andrews Cluster Wardlaw
Molecular QuantumChemistry
GaussianNWChemGAMESS-UKGAMESS-USORCADALTONMOLPROTurbomole
Application Areas and Programs
Plane Wave DFTCASTEPVASPCPMDNWChemPWscf
Molecular MechanicsGULPNWChemDL_POLY
Other Periodic MethodsWien2kCRYSTALSIESTACP2K
Connecting from Windows:XMing and PuTTY
InstallationXMing:
Download fromhttps://sourceforge.net/projects/xming/and install(in case of graphics problems later, get Xming-mesa fromhttp://www.straightrunning.com/XmingNotes/)
PuTTY
Download Windows installer fromhttp://www.chiark.greenend.org.uk/~sgtatham/putty/download.htmland execute it
Starting PuTTY Terminal
Start Xming (runs in
background)
Start PuTTY
Make sure X11 forwardingis enabled
Can save session settingsand recall
Specify host in Sessionwindow
Start terminal
File TransferUsing cygwin (or Linux) command line:
sftp <user>@wardlaw10.mcs.st-andrews.ac.uklogin, password…put <file> PC -> wardlawget <file> wardlaw -> PCexit
Alternative: WinSCP
http://winscp.net/eng/index.php
graphical user interface (drag and drop)
Convert text files (newline character differs between UNIX and DOS/Windows)
dos2unix <file> Windows -> Linux both commands should workunix2dos <file> Linux -> Windows on cluster and in CYGWIN
Not doing this can lead to weird errors!
Creating and Using ssh KeysOn Linux, Mac or Cygwin:
Enter commandssh-keygen
Accept all default filenames
Choose (and remember) passphrase when prompted
Email file ~/.ssh/id_rsa.pub to system administrator.(In Cygwin, this is C:\cygwin64\home\<username>\.ssh\id_rsa.pub)
Do NOT send the private key (id_rsa without suffix)
Creating ssh Keys with PuTTYgen
PuTTYgen • Start PuttyGEN (in PuTTYfolder in start menu).
• Select "Generate"
• Move mouse randomlythrough grey area in windowuntil finished.
• Save private key. You willneed this to connect.
• Copy long string in upperwindow into text file.
• Email this file to sysadmin.
Using ssh Key with PuTTY
PuTTY
• Select Auth tab underConnection -> SSH
• Use browser menu to findprivate key (.ppk)
You can save settings (key location and X11 enabling in theSession tab.
Using ssh Key with WinSCP
• Select Advanced settings
• SSH -> Authentication tab
• Use browser menu to find private key (.ppk)
Linux commandsman <command>
manual page for <command>
exit
quit current shell (log out if this is the login shell)
passwd
change password
use a cryptic one!
Not necessary on wardlaw, kennedy or knox(authentication is via ssh keys).
Linux commands
ls
list current directory
ls -l
list directory in detail
ls -lrt (same as ls -l -t -r)
list current directory in detail,ordered by time, in reverse order
ls <dir>
list directory <dir>
man ls ...
more Linux commands
cd <directory>
“change directory”: change working directory (folder)
.. parent directory (one up in the tree)
. current directory
/ between parent and child (not \ as in Windows)
pwd“print working directory”: show where we are
mkdir <directory>
create directoryrmdir <directory>
delete (empty) directory
Still more Linux commands
cp <file1> <file2>
copy
mv <file1> <file2>
rename (“move”)
mv <file1> <dir>
move <file1> into directory <dir> (works also with cp)
rm <file>
delete <file>
rm -r <dir>
delete <dir> and all subdirectories!
A few more useful ones...head <file>
show first 10 lines of file
head -20 <file>
show first 20 lines (works for any number)
tail <file>
show last 10 lines (e.g. check if output file is complete)
tail –20 <file>
guess what…
tail –f <file>
“follow”: keep showing newly added lines. Exit with Ctrl-c
And the last one for now
grep <pattern> <file>
show all lines in <file> containing <pattern>
Wildcards Most file/directory commands accept “wildcard
characters”. Most commonly used ones are
? Any single character
* Any number of characters (including 0)
Editing with Emacs
emacs <file> &
Opens Emacs window on your desktop
& executes (any) program in background, soyou can go on working in the same window
Using Emacs
Menus are self-
explanatory
Try to remembersome keyboardshortcuts, in case youneed to use it withoutX client (emacs -nw)
Ctrl-x-s save
Ctrl-x-c exit
Ctrl-x-w save underdifferentname
Editing with vi Faster over a network
No separate window start with
A bit cryptic at first… vi <file>
Two modes
Command Mode: cursor keysand 1-2 letter commands
x delete character
dd delete line
D delete rest of line
i insert before cursor
a append after cursor enter
o new line below input
O new line above mode
:w save
:x save and exit
Input Mode
Write text
Delete with backspace key
[ESC] key: back to command mode
Many tutorials and “cheatsheets” available
Google “vi editor”
Running SLURM Batch Jobs
Submit script to batch queuefor execution when computenodes are available
Job script with SLURMdirectives at the start:
Name of job
Request one nodeUse 12 coresRun for 48h maxUse complete nodesScript to run Gaussian
SLURM directives
Commands toexecute when jobstarts
Managing jobs
sbatch <jobscript>submit job
squeueshow running or queued jobs
scancel <jobnumber>delete job
Other Queueing Systems
Other clusters (knox, chpc-hf63):SGE or PBS queueing system
sbatch => qsubsqueue => qstatscancel => qdel
Get an example job from your supervisor!
Privacy and Data Protection
UNIX File permissions:
user (=owner)
group (same group as owner)
others (anybody else)
read
write
execute (file) or access (dir)
To change:
chmod go-rwx
(remove all permissions from yourgroup and others)
chmod g+r
(give your group read permission)
man chmod …
Root user can read (or write) anything!
Security
Passwords
• Don’t share. Get your ownaccount
• Make it difficult to guess(no names, dictionarywords, birth dates; usespecial characters)
• Change if compromised
Don’t run unknown programs
Don’t log in from a PC thatmay have viruses
Rules of Use
Wardlaw and kennedy open to any St Andrewschemist
Higher priority access available for £500/y
Be considerate: don’t flood the machines whenthey are already full.
Other clusters: ask your supervisor
How to get an Account or Advice
Contact me!
Wardlaw website
http://www-solar.mcs.st-and.ac.uk/~herbert/cluster/
Go down to “Software and usage”.
EaStCHEM/SUPA postgrad course Introduction to Computational Chemistry
9 lectures starting October
Email your name and supervisor to Tanja van Mourik
Other Lectures and SeminarsLectures on
Molecular Quantum Chemistry
DFT on Periodic Systems
Computation in Chemistry Seminars
See Chemistry Newsletter
Sign up to mailing list with Tanja van Mourik
EaStCHEM PgCert course Computational Chemistry and Modellinghttp://www.ed.ac.uk/studying/postgraduate/degrees/index.php
Free for St Andrews PG students