introduction - defining · pdf fileintroduction - defining unix ... built on free softwre...

INTRODUCTION - DEFINING UNIX

****************************

Unlike say IBM’s OS390, Microsoft DOS or Windows or DEC (Compaq) VMS,

UNIX is not an Operating System, offered by a single vendor, to run on

a given hardware architecture. It is a generic Operating Environment.

To remain precise, UNIX represents a family of Operating Systems,

which, like the genetic sort, share a lot of features and mechanisms,

but also show some variations. Right from the beginnings in 1970, new

lines have kept appearing, while others have become obsolete. Much the

same can be said of Linux, the major contemporary PC flavour of UNIX.

These traits may be grouped according to their historical roots, but

most current UNIX implementations have encompassed many features from

all these backgrounds. In fact at the User level, one can work on

UNIX without needing to become aware of these peculiarities.

Major Components in UNIX :

--------------------------

- 1/ Files - Generally, Recipients for Data,

owned and controlled by Operating System or User(s).

Special Types : Directories (Groups of File Names),

Links (Pointers to Files),

Devices (Channels to Hardware).

- 2/ Commands - Request Tasks.

May number over 2000, though about 20 can suffice.

- 3/ Shells - Command Interpreters,

operating in Interactive or Batch Mode,

latter via submission of Scripts.

- 4/ X - Protocol to build and run Graphical Interfaces,

in a Networked (Client/Server) environment,

where Sessions handled by Window Managers, or,

in more recent terminology, "Desktop Environments".

2

UNIX Versions (1)

*****************

Unix Definitions (Specifications) :

===================================

System V :

Last major revision from ATT (Bell Labs),

who had started from failure of Multics in early 1970s.

System V Release 4 was last version before end of ATT effort.

BSD (Berkeley Software Development) :

Arose from work on Berkeley Campus of University of California,

and directly led to genesis of SunOS, first Sun implementation,

but influenced all Unix variants from mid 1970s on,

except SCO a strict System V Release 3, and now in disuse.

OSF/1 (Open Software Foundation) :

Attempt, led mostly by DEC and IBM, at producing a Standard,

incorporating best features of both System V (Release 4) and BSD.

Linux :

Generic name for (mostly Personal Computer) implementations,

built on Free Softwre Foundation Kernel written by Linus Torvald.

Linux "Distributions" often free or comparatively cheap, and

differing mainly in User Interfaces and set up procedures.

3

UNIX Versions (2)

*****************

Unix Implementations (Specific Versions) :

==========================================

AIX : Advanced Interactive Executive, IBM’s commercial Unix,

for RS Series hardware, from 1990,

now on Mainframes and Supercomputers too.

DEC Unix : OSF/1 (strict) commercial version from DEC,

now fallen in disuse (with DEC).

HP-Ux : Hewlett Packard commercial version.

Irix : Silicon Graphics commercial version,

now merely an alternative to Linux on newer SGI hardware.

Linux : Many distributions, like Debian, Fedora, Red Hat or Suse,

but also offered by manufacturers like HP, IBM, even Sun,

in IBM’s case, as a mainframe alternative.

MacOS-X : Apple now builds its Mac software on Unix underpinnings,

which may be put to use under usual Mac Interface.

SCO : Santa Cruz Operations, who bought rights to System V,

but failed to get system widely adopted.

Solaris : Sun’s System V implementation, on Sparc or Intel hardware,

has mostly superceded SunOS.

SunOS : Sun’s original Unix set up, strictly BSD.

Ultrix : DEC’s initial Unix version, now defunct.

Xenix : Microsoft’s Unix version, developed soon after DOS,

and offered until mid 1980s, before Windows took off.

4

UNIX, ROOTS AND SUBSEQUENT IMPLEMENTATIONS

******************************************

5

Generic Unix "File System" Directory Tree :

*******************************************

/ (Root) --+-- /bin Essential Operating Binaries (1)

|

+-- /dev Hardare Channels (Device Files)

|

+-- /etc System Configuration Files

|

+-- /home User Accounts (Home Directories),

| possibly numbering in thousands

|

+-- /local Local (machine specific) Software

|

+-- /lost+found Area for damaged Files

|

+-- /mnt Dynamic

|

+-- /opt Optional Software and Files

|

+-- /proc Run Time Process (Program) Images

|

+-- /sbin Essential Opearting Binaries (2)

|

+-- /tmp Primary Scratch Data area,

| accessible to any User on System

|

+-- /usr ----+-- /bin Common Command Binaries

| +-- /local Special Programs

| +-- /man Help (’man’) Files

| +-- /sbin Static Binaries

| +-- /share Platform independent Software

| +-- /X11 Windowed Interface Files

|

+-- /var ----+-- /adm Admin Data and Logs

+-- /log System or Admin Logs

+-- /mail Email Files (1)

|

+-- /spool --+-- /cron Automated Job Control

| +-- /lpd Print Control and Data

| +-- /mail Email Files (2)

|

+-- /tmp Secondary Scratch area

6

UNIX FILE NAMES (1)

*******************

File Names can take as few as 1 Characters, and usually up to 256.

Names cannot include Slash (’/’) Characters, except for Directories.

Most other Keyboard Characters can go in, but some should be avoided,

as these non Alphanumerics take special meanings on Command Lines,

which hinders either File Name or Command interpretation.

Characters to avoid in File Names :

===================================

char Description Special Use (by Shell)

---- ------------------------------------- -------------------------------

’*’ Asterisk File Name Wild Card,

for any number of Characters.

’?’ Question Mark File Name Character Wild Card,

for any Single Character.

’ ’ Space Separator in Commands.

’@’ ’At’ Sign Mail Addresses Separator.

’#’ Hash, or ’Pound Sign’ for Americans Make rest of Line a Comment.

’$’ US Dollar Sign, Environment Variable Prefix.

’^’ Caret or Circumflex Search Expressions Prefix.

’&’ Ampersand Execute Command in Background.

’()’ Parenthesis Group Commands on Line.

’[]’ Square Brackets Anyone of Characters enclosed.

’\’ Backslash Continue Command on next Line.

’;’ Semi-Colon Command Separator on Line.

’|’ Vertical Bar, or Pipe Symbol Output to next Command on Line.

’>’ ’Greater than’, or Redirection Symbol " " File after Sign.

’<’ ’Less " ’, " " " Input from " " " .

’!’ Exclamation Mark Repeat Prefix for some Shells.

’~’ Tilde (or ’Twiddle’) Home Directory Symbol.

’‘’ Back Quote Command Output as Operand.

’"’ Double " Character String.

’’’ Single " (or Apostrophe) " " (Literal).

Likewise, all Non Display Characters (Carriage Returns, Tabs, etc.)

must be avoided, as they remain unseen on the Command Line or in Output,

and cause unpredictable behaviour when Command interpreted.

7

UNIX FILE NAMES (2)

*******************

Neutral non Alphanumeric Characters for File Names :

====================================================

char Description Restrictions - Comments

---- ------------------- -----------------------------------------------

’-’ Hyphen (Minus Sign) Cannot be First Character (clash with Options).

’.’ Dot (Full Stop) When First Character, "hydes" File to ’ls’,

but not to ’ls -a’ or ’ls -al’.

’_’ Underscore

’:’ Colon

’+’ Plus Sign

’=’ Equal Sign

File Name Wild Cards :

======================

char Description Use

---- ------------------ ----------------------------------------------------

’*’ Asterisk (or Star) Any Character, including none,

like ’res*.dat’ for all Files with ’res’ as Prefix,

and ’.dat’ as Suffix (or Extension).

’?’ Question Mark Any Single Character,

like ’???’ for all Files with 3 Characters in Name.

’[]’ Square Brackets Any Single Character from Set inside Brackets,

or if Set preceded by ’^’ (Caret), not from Set,

like ’ex[ab].dat for ’exa.dat’ and ’exb.dat’,

or ’ex[^ab].dat’ for Files without ’a’ or ’b’,

as third Character in Name.

For Files with Special Characters (or Spaces) in Name, can place verbatim

Name between Double Quotes ("), or (more restrictive) Single Quotes.

File Name Examples :

====================

’/’ = Root (Base) Directory of File System.

’.’ = Short Hand for Current Directory in Session.

’..’ = " " " next " below Current (’.’).

’/usr/bin/ls’ = A typical Command Module.

’Test.dat’ = Some Test Data.

’"Gap Name"’ = Using Double Quotes to handle Space in Name.

8

UNIX COMMANDS (1)

*****************

General Form :

--------------

’Command_name -Options Operands’

Where :-1/ Command Name may be Prefixed with Directory Name(s),

which represents Directory where (Binary) File resides.

This is needed when Dir. not in PATH Environment Variable.

e.g. ’/bin/ls’ for Command ’ls’.

-2/ Options usually Single letters,

either each with ’-’ (hyphen) Prefix, and Space Separated,

or as a group, with single ’-’ (hyphen) Prefix,

but sometimes also include Digits or Double Quoted Strings.

e.g. ’ctags -u -f NewFile OldFile’

’ls -alg /bin/’

’cut -d" " -f1 inp_File’

-3/ Operands often one or more File Names (possibly Wild Carded),

but sometimes not needed, User Names or, Character Strings.

e.g. ’rm File_1 File_2 File_3 File_4’

’echo "This will show on Screen"’

’ls -al paper*.tex’

’lpr -P topn chap[01234].ps’

9

UNIX COMMANDS (2)

*****************

Special Characters in Commands :

--------------------------------

char Description Use or Effect

---- ------------------- ---------------------------------------------

’;’ Command Separator To Enter several Commands on same Line,

like ’pwd ; ls -al’ (treat with respect).

’\’ Line Continuation To finish typing Command on next Line,

when (and only when) Last Character on Line,

and so on every time Keyed again,

like ’ls File_1 \’

then ’ File_2 ’, after ’>’ System Prompt,

or on next Script Line.

’|’ Pipe Command Output So Command Output (File or other),

Input to (next) Command following Symbol,

like ’cat File_1 | grep Data_Entry’.

’&’ Background Execute Forces Command to run non Interactively,

when spare Processor Time avails,

and gives immediate Prompt for next Command,

like ’xterm &’ (new ’xterm’ Command Window).

’<’ Input Redirection Command takes Input from File, not Keyboard,

like ’dc < Inp_File’.

’>’ Output " (1) Command sends Output to File, not Screen,

like ’cat Inp_File.* > Out_File’.

’>>’ " " (2) Command appends Output to File, not Screen,

like ’cat Inp_File.* >> Add_to_End’.

’2>’ Error " Command sends Messages to File, not Screen,

like ’ls -al *.for 2> Error_Log’.

10

UNIX COMMANDS (3)

*****************

Investigative Commands :

------------------------

Commands to extract Information from Files or Operating System,

with no risk of accidents or adverse effects (Passive Commands).

Main Commands : ’cd’

’pwd’

’man’

’ls -al’ also ’ls -alR’, ’ls -alL’

’which’

’file’

’cat’

’more’

’head’

’tail’

’grep’ or ’cat File | grep "Data"’

’find’

’cmp’

’dircmp’ not on all implementations

’lp’ also ’lpr’

’df -k’

’du -k’ or ’du -ks’

’ps -aux’ (BSD) or ’ps -ef’ (System V)

Adaptive Commands :

-------------------

Commands Creating, Amending or Deleting Files or Variables on System,

implying risk of accidental changes in so doing (Active Commands).

Main Commands : ’cp’ also ’cp -rp’

’ln’ also ’ln -s’

’mv’

’rm’ also ’rm -r’

’mkdir’

’rmdir’

’chmod’ also ’chmod -R’

11

LIST ’ls -al’ COMMAND OUTPUT (1)

********************************

A User’s Home Directory (Command ’ls -al "$HOME"’) :

=====================================================

TPerm. Ln. Owner Grp. Size Date Time File Name

---------- --- ----- ---- ---- ------ ----- -------------------------------

total 210

drwxr-xr-x 20 oper loc 1024 Oct 22 17:46 .

dr-xr-xr-x 24 root root 512 Oct 6 17:24 ..

-rw------- 1 oper loc 445 Oct 22 17:46 .Xauthority

lrwxrwxrwx 1 oper loc 23 Jul 9 16:00 .Xdefaults ->

/opt/adm/nus/.Xdefaults

-r-xr-xr-x 1 oper loc 55 Oct 2 1996 .cshrc

lrwxrwxrwx 1 oper loc 18 Mar 11 1997 .exrc -> /opt/adm/nus/.exrc

lrwxrwxrwx 1 oper loc 20 Mar 11 1997 .kalias -> /opt/adm/nus/.kalias

lrwxrwxrwx 1 oper loc 19 Mar 11 1997 .login -> /opt/adm/nus/.login

lrwxrwxrwx 1 oper loc 26 Mar 11 1997 .openwin-menu ->

/opt/adm/nus/.opwn-menu

-rw------- 1 oper loc 68736 Oct 22 20:20 .sh_history

lrwxrwxrwx 1 oper loc 19 Mar 11 1997 .twmrc -> /opt/adm/nus/.twmrc

-r-xr-xr-x 1 oper loc 1016 Jul 14 20:59 .userlogin

lrwxrwxrwx 1 oper loc 20 Mar 11 1997 .winmgr -> /opt/adm/nus/.winmgr

-rw-r--r-- 1 oper loc 29 Jul 22 21:49 .xinitrc

-r-------- 1 oper loc 44 Oct 22 17:46 .xsun.host:0

-rwxr-xr-x 1 oper loc 289 Oct 22 17:46 .xtermsrc

drwxr-xr-x 2 oper loc 512 Jul 23 22:55 bin

drwxr-xr-x 9 oper loc 512 Dec 5 1996 ccc

dr-xr-xr-x 2 oper loc 512 Apr 30 21:41 cms

dr-xr-xr-x 2 oper loc 1024 Apr 30 21:40 cob

dr-xr-xr-x 7 oper loc 512 Oct 12 18:15 crn

drwxr-xr-x 8 oper loc 1024 Oct 1 22:20 doc

drwx------ 4 oper loc 512 Oct 10 10:27 ecb

drwxr-xr-x 5 oper loc 512 Mar 26 1996 for

drwxr-xr-x 4 oper loc 512 Jul 9 1996 glm

dr-xr-xr-x 3 oper loc 2560 Apr 30 21:41 ims

dr-xr-xr-x 7 oper loc 512 Jul 9 1996 msc

dr-xr-xr-x 2 oper loc 512 Apr 30 21:41 pli

dr-xr-xr-x 4 oper loc 1024 Apr 30 21:41 rpt

drwxr-xr-x 4 oper loc 1024 Jun 1 18:58 scr

drwxr-xr-x 2 oper loc 1024 Jul 9 1996 sss

drwxr-xr-x 3 oper loc 1024 Jul 10 17:44 tex

drwxr-xr-x 5 oper loc 512 Jul 19 1996 utl

drwxr-xr-x 5 oper loc 1024 Oct 14 18:05 wrk

12


********************************

A DAMTP Home Directory (Command ’ls -al "$HOME"’ or ’ls -al ~’) :

==================================================================

TPerm. Ln. Owner Group Size Date Time File Name

---------- --- ------ ----- -------- ------ ----- ----------------

total 185

drwxr-xr-x 15 ecb34 other 1024 May 7 19:11 ./

drwxr-xr-x 70 root root 1536 Apr 30 20:53 ../

-rwxr-xr-x 1 ecb34 other 2038 Feb 8 10:16 .Terms*

-rw------- 1 ecb34 other 12535 May 7 19:09 .Xauthority

-rw-r--r-- 1 ecb34 other 6983 Dec 28 19:56 .Xresources

-rwxr-xr-x 1 ecb34 other 2194 Jan 1 20:06 .Xterm*

-rw-r--r-- 1 ecb34 other 1601 Apr 28 16:47 .acrorc

-rw-r--r-- 1 ecb34 other 0 Dec 6 14:47 .addressbook

-rw-r--r-- 1 ecb34 other 2285 Dec 6 14:47 .addressbook.lu

-rw------- 1 ecb34 other 571 May 7 19:10 .bash_history

-rw-r--r-- 1 ecb34 other 859 Dec 6 12:22 .bash_logout

-rw-r--r-- 1 ecb34 other 1167 Dec 6 12:22 .bash_profile

-rw-r--r-- 1 ecb34 other 1510 Dec 30 20:18 .bashrc

-rw-r--r-- 1 ecb34 other 1138 Dec 6 12:22 .cshrc

-rw-r--r-- 1 ecb34 other 1083 Dec 6 12:22 .emacs

-r--r--r-- 1 ecb34 other 2052 Dec 6 18:44 .exrc

-rw-r--r-- 1 ecb34 other 24787 Jan 2 19:19 .fvwmrc

-rw------- 1 ecb34 other 25236 Jan 1 15:48 .fvwmrc.bak

-rw------- 1 ecb34 other 36 Jan 26 10:16 .history

-rw-r--r-- 1 ecb34 other 1138 Dec 6 12:22 .login

-rw-r--r-- 1 ecb34 other 849 Dec 6 12:22 .logout

-rw-r--r-- 1 ecb34 other 665 Dec 14 10:54 .mailrc

-rw-r--r-- 1 ecb34 other 0 Dec 23 12:58 .motdrc

drwxr-xr-x 4 ecb34 other 512 May 3 13:06 .netscape/

-rw-r--r-- 1 ecb34 other 10327 May 7 19:11 .pinerc

drwxr-xr-x 2 ecb34 other 512 Dec 19 18:03 .ssh/

-rw-r--r-- 1 ecb34 other 16936 Jan 2 18:58 .twmrc

-rw------- 1 ecb34 other 16936 Mar 24 18:31 .twmrc.bak

-rwxr-xr-x 1 ecb34 other 3017 Feb 8 10:11 .xsession*

-rw-r--r-- 1 ecb34 other 887 May 7 18:45 .xsession-errors

drwxr-xr-x 21 ecb34 other 512 Dec 9 17:54 Dir/

drwxr-xr-x 3 ecb34 other 512 May 3 19:02 HEP/

drwxr-xr-x 2 ecb34 other 512 Mar 23 10:30 NOBACKUP/

drwxr-xr-x 4 ecb34 other 512 Apr 18 16:49 SYS/

drwx------ 2 ecb34 other 512 May 6 17:42 mail/

drwx------ 2 ecb34 other 512 Feb 16 17:14 nsmail/

drwxr-xr-x 3 ecb34 other 512 Mar 14 17:16 public_html/

drwxr-xr-x 3 ecb34 other 2048 May 7 19:22 wrk/

13


********************************

’ls -al "$HOME"’ Command Notes :

----------------------------------

Legend : -1/ ’T’ = File Type : ’d’ = Directory,

’l’ = Symbolic Link,

’-’ = Normal (Data or Program).

Other Characters indicate Special File.

-2/ ’Perm.’ = Permissions, or Access Rights,

in 3 Sets of 3 Characters,

for ’Owner’, ’Group’ and ’other’,

each ’Read’, ’Write’, ’Execute’,

when ’r’, ’w’, ’x’ or ’-’ when not.

-3/ ’Ln.’ = Link Count (Hard Links).

-4/ ’Owner’ = User Name of File Owner.

-5/ ’Group’ = Group " " " " .

Note : -1/ ’Alphabetical’ Ordering of File Names,

with ’.’ (Hidden) Files first (not shown without ’-a’ Option),

including ’.’ for Current Directory and ’..’ for that below,

then Capitals coming before Lower Case Letters.

-2/ Dates and Times refer to last Change to File contents,

with Year replacing Time when over 6 Months old.

-3/ ’->’ points to actual File in Symbolic Links,

which can be shown with ’ls -alL’ Command.

-4/ To expand Sub-Directories, use ’ls -alR’ Command.

-5/ To append File Type Flag to to Name, use ’ls -alF’ Command,

so, ’/’ follows Directories, and ’*’ Executables in output.

’F’ Option a default on some implementations (example 2).

-6/ As these examples show Home Directories,

Standard Session Initialisation Files present,

in first case mostly as Symbolic Links to System defaults.

Also note no Data Files, only Directories for neatness.

-7/ Most Shells accept ’~’ (Tilde) for ’"$HOME"’ above.

14

ACCESS CONTROL ’chmod’ COMMAND (1)

**********************************

UNIX File Access Control :

--------------------------

For UNIX Files : 3 Access levels : Read (Letter Code ’r’),

Write ( " " ’w’),

Execute ( " " ’x’).

3 Access Categories : Owner or User (Letter Code ’u’).

Group ( " " ’g’),

Other or World ( " " ’o’).

Checking current Access Controls (Permissions) :

------------------------------------------------

In output from ’ls -al’ List Command, Access Levels, or Permissions,

appear as 3 triplets of Codes in positions 2 to 10 of Left Hand Column.

In order, Triplets indicate ’User’, ’Group’ and ’Other’ Permissions,

and within each Triplet, ’Read’, ’Write’ and ’Execute’ Access,

so a given Character always stands for same Category and Access Level.

Set Permissions show as their letter Codes, disabled ones as Hyphens (’-’).

Special Access Control requirements :

-------------------------------------

Compiled Binaries : Execute (’x’) only needed.

Shell Scripts : Read (’r’) and Execute (’x’) needed,

when Script submitted as a Command.

Read (’r’) alone needed,

when Script File input to Current Shell,

as if typed at Terminal.

Directories : Read (’r’) to see contents (run ’ls -al on Dir.).

Write (’w’) to change contents,

that is, create or delete Files,

thus preventing deletions when not set,

but not necessarily changes to Files.

Execute (’x’) to access or execute individual Files,

when their Names known.

15


**********************************

Expressing Permissions with Octal Digits :

------------------------------------------

Permission Triplets can be written as 3 Digit Binary Numbers, where,

first digit stands for ’Read’, Second for ’Write’, Third for ’Execute’,

with ’1’ for set Permissions and ’0’ otherwise (’-’ showing in ’ls -al’).

Such Binary Numbers can be written as one Digit Octal Numbers,

as they range from ’0’ (actually ’000’) to ’7’ (’111’ in Binary).

So, full set of 3 Permissions Triplets can be written as 3 Octal Digits.

Example : Category User Group Other

Permissions r w x r - x r - -

Binary 1 1 1 1 0 1 1 0 0

Octal 7 5 4

Alternatively, Weights (Decimal or Octal) can be given to Permissions :

Category User Group Other Note

Permission r w x r w x r w x

| | | | | | | | | Weight

| | | | | | | | +---- 1 Third Digit only,

| | | | | | | +------ 2 for ’Other’ Perms.

| | | | | | +-------- 4

| | | | | |

| | | | | +----------- 10 Second Digit only,

| | | | +------------- 20 for ’Group’ Perms.

| | | +--------------- 40

| | |

| | +------------------ 100 First Digit only,

| +-------------------- 200 for ’User’ Perms.

+---------------------- 400

Latter expression provides same result as Octal mode,

or as if full 3 Triplets set taken as a 9 Digit Binary Number.

Example above would give : 400 + 200 + 100 + 40 + 10 + 4 = 754

16


**********************************

Default Permissions :

---------------------

When Files created, all possible Permissions set (giving ’777’ in Octal).

Files not recognised as Binary Executables then loose Execute Permissions,

and a Mask, set with ’umask NNN’, subtracted, where NNN an Octal Triplet.

Entering ’umask’ alone shows current Mask Value in Octal.

Typically, Mask ’022’ (withdraw Write Permission for ’Group’ and ’Other’),

so a non Binary’s initial ’666’ becomes ’644’ or ’rw-r--r--’ in ’la -al’.

Examples : New Text File starts with ’666’ or ’rw-rw-rw-’,

less Mask ’022’ or ’----w--w-’,

=== =========

giving ’644’ or ’rw-r--r--’.

New Binary starts with ’777’ or ’rwxrwxrwx’,

less Mask ’022’ or ’----w--w-’,

=== =========

giving ’755’ or ’rwxr-xr-x’.

17


**********************************

Changing File (Directory) Permissions - ’chmod’ Command :

---------------------------------------------------------

Syntax : ’chmod new_permissions files(s)’ (’-R’ Option for Recursive).

’chmod’ operates in 2 Modes : Absolute - Force a set of Permissions,

regardless of current settings,

always with 3 Digit Octal Number.

Relative - Set, Add or Withdraw Permissions,

from existing settings,

via Category and Perm. Codes.

Adding an already set Permission, or removing an absent one has no effect.

In Absolute Mode, New Permission simply presented as 3 Octal Digits.

In Relative Mode, Changed Permissions given as ’who-how-what’ with :

’who’ = Category of User affected (or ’a’ for ’All’),

’how’ = Action : ’=’ to Set (no ’what’ cancels all),

’+’ " Add,

’-’ " Withdraw.

’what’ = Permissions affected : ’r’ = ’Read’,

’w’ = ’Write’,

’x’ = ’Execute’.

examples : ’chmod 444 tst1 tst2’ : All Categories forced ’Read’ only.

’chmod 755 *.bin’ : Give ’Read-Write-Execute’ to ’User’,

and ’Read-Execute’ to ’Group’, ’Other’.

’chmod go-w *.bin’ : Take out ’Write’ from ’Group’, ’Other’,

in effect, as above. Note Wild Card.

’chmod ugo+x scrp’ : Add ’Execute’ to all Categories.

Note Script Files need this applied.

’chmod a+x scrp’ : Category Wild Card form of above.

18

UNIX SEARCHING - REGULAR EXPRESSIONS (1)

****************************************

Purpose :

---------

Regular Expressions (REs) let many UNIX Editors, and Filters pick Text,

from ASCII (not Binary) Files, using special signs to specify Patterns.

Text is scanned Line by Line against one or more Expressions.

Expression syntax can be very elaborate, even for UNIX experts,

and complex Expression matchings against large Files very lengthy.

Only an outline of basic forms of Expressions is given here.

Basic Special Characters in Regular Expressions :

=================================================

char Description Special Use

---- --------------------- ------------------------------------------

’.’ Dot (Full Stop) Any Single Character.

’?’ Question Mark Zero or one of Char. or Expr. to left.

’*’ Asterisk Any number " " " " " "

’+’ Plus Sign At least one " " " " , in ’egrep’.

’^’ Caret or Circumflex If First, match at Line’s Start,

or Negation when first in ’[]’.

’$’ US Dollar Sign When Last, match at Line’s End.

’[]’ Square Brackets Anyone of Characters enclosed,

or none of when ’^’ First Char.

’-’ Hyphen or Minus Sign Indicate Range inside ’[]’ Pair.

’\’ Backslash Escape, for Verbatim Special.

’|’ Vertical Bar Match Expr. on one side of ’|’ in ’egrep’.

Presentation :

--------------

Characters without special meaning just matched verbatim in search,

but unless pure text, without Special Characters for Shell or UNIX,

enclose Expressions given on Command Line inside Single Quotes.

19

UNIX SEARCHING - REGULAR EXPRESSIONS (2)

****************************************

’Text’ : All Lines with Word ’Text’.

Note Quotes not necessary here.

’^my data’ : Lines starting with ’my data’.

’ [1-9][0-9]*$’ : Lines ending with a Space and at least one Digit.

’[0-9]* [0-9]*$’ : Lines ending with ’Digit(s)-Space-Digit(s)’.

Note this would match above Expression.

’^ *’ : Lines starting with any number of Spaces.

’^[Cc*]*’ : Lines starting with any number of ’C’, ’c’, ’*’.

’ Taxi | Bus ’ : Lines with Words ’Taxi’ or (and) ’Bus’ in ’egrep’.

’ [a-zA-Z]*\.\* ’ : Lines with ’Space-Letters-.-*-Space’,

like File Names with Dot and Wild Carded Extension.

Note 2 Ranges inside ’[]’ and Escaped ’.’ and ’*’.

’ [^a-zA-Z0-9] ’ : Lines with non Alpha Numeric Text Strings.

Note 3 Ranges inside ’[]’ Pair, after ’not in’ Sign.

Expressions in Commands :

-------------------------

"ypcat passwd | grep Elie" : Lines with ’Elie’ in Passwords File.

"cat /etc/aliases | grep ’GR|HEP’" : Lines with either ’GR’ or ’HEP’

in email Aliases File.

"grep ’ *[0-9]* FORMAT’ pr*.for’ : ’FORMAT’ Statements in Fortran Files.

Note ’grep’ used without piped input,

so will show File Name of Match.

20

UNIX DISK STATUS COMMANDS (1)

*****************************

On System V Systems : ’df -k’ = Disk Partitions usage Data (KBytes).

’du -k’ = Directory Space " " ( " ).

On some BSD Systems, ’-k’ Option redundant (or invalid).

’df’ Output :

-------------

Filesystem kbytes used avail capacity Mounted on

/dev/dsk/c0t0d0s0 31239 12149 15970 44% /

/dev/dsk/c0t0d0s3 122695 73731 36704 67% /usr

/proc 0 0 0 0% /proc

fd 0 0 0 0% /dev/fd

/dev/dsk/c0t0d0s4 61735 5342 50223 10% /var

swap 698160 488 697672 1% /tmp

/dev/dsk/c0t0d0s5 246167 88559 132998 40% /usr/openwin

/dev/dsk/c0t0d0s6 914542 320598 502494 39% /opt

/dev/dsk/c0t0d0s7 451183 281114 124959 70% /opt/gen

/dev/dsk/c0t1d0s0 914542 108138 714954 14% /export/home

/dev/dsk/c0t2d0s0 914542 546396 276696 67% /export/ima1

/dev/dsk/c0t3d0s0 914542 543984 279108 67% /export/ima2

atlas:/export/home 1829096 1510272 135912 92% /nfs_mnt/atlas/home

atlas:/export/ima1 1845768 1030840 630352 63% /nfs_mnt/atlas/ima1

atlas:/export/ima2 1845768 1312872 348320 80% /nfs_mnt/essex/ima2

essex:/opt/gen 468798 205027 216901 49% /opt/mnt

Note : -1/ ’Filesystem’ shows actual Device (’/dev/’) Name when Local,

or ’Host_Name:Path’ when external (exported) Partition.

Device Name structure varies between implementations,

above example Sun Solaris (and ATT System V Release 4).

-2/ ’used’ and ’avail’ Sum to less than ’kbytes’,

as File System (Partition) management hard when near full,

so it is claimed as full at threshold of loss of performance.

-3/ ’capacity’ Column gives Space Used in per cent.

-4/ Add Option ’-l’ for Local Partitions only.

-5/ Note ’df’ only shows Partitions mounted (in use) at that time.

In networked context, these may appear on demand, then time out.

21

UNIX DISK STATUS COMMANDS (2)

*****************************

’du’ Output (’du ./for’) :

--------------------------

35 ./for/gen

373 ./for/apl

996 ./for/mri

1405 ./for

Note : -1/ Directory may contain Files as well as Sub-Directories,

Hence Total given may exceed Totals for Sub-Directories.

-2/ Add Option ’-s’ for Overall Total only,

for instance ’du -s $HOME’ (or ’du -ks ~’) for all own Files,

like ’$ du -ks ~’ gives about 45 MBytes in my Home Dir. :

’45209 /home/raid/supp/ecb34’

-3/ Tally will only include Directories visible to User.

Error Messages will point to hidden Directories,

of which Disk usage not made part of computed Total.

22

UNIX SYSTEM STATUS COMMANDS (1)

*******************************

UNIX System V ’ps -ef’ Command Output :

------------------------------------------

Owner Proc. Parent C Start Terminal CPU Command

(PID) (PPID) Time Time

------ ------ ------ --- -------- -------- -------- ---------------------

root 0 0 0.0 Apr 29 ?? 49:10.01 [kernel idle]

root 1 0 0.0 Apr 29 ?? 0:01.60 /sbin/init -a

root 3 1 0.0 Apr 29 ?? 0:00.05 /sbin/kloadsrv

root 22 1 0.0 Apr 29 ?? 6:12.09 /sbin/update

root 98 1 0.0 Apr 29 ?? 0:01.35 /usr/sbin/syslogd

root 279 1 0.0 Apr 29 ?? 0:09.92 /usr/sbin/portmap

root 287 1 0.0 Apr 29 ?? 0:01.31 /usr/sbin/ypbind -s

root 296 1 0.0 Apr 29 ?? 1:59.33 /usr/sbin/mountd -i

root 298 1 0.0 Apr 29 ?? 0:00.02 /usr/sbin/nfsd -t8 -u

root 303 1 0.0 Apr 29 ?? 36:47.38 /usr/sbin/rpc.statd

root 305 1 0.0 Apr 29 ?? 0:06.43 /usr/sbin/rpc.lockd

root 399 1 0.0 Apr 29 ?? 0:14.21 /usr/sbin/sendmail -q

root 402 1 0.0 Apr 29 ?? 0:10.53 /usr/sbin/xntpd -g -c

root 424 1 0.0 Apr 29 ?? 0:02.20 /usr/sbin/inetd

root 431 1 0.0 Apr 29 ?? 0:00.81 /usr/sbin/cron

root 445 1 0.0 Apr 29 ?? 0:07.51 /usr/lbin/lpd

root 451 1 0.7 Apr 29 ?? 10:18.67 amd -F /etc/amd.conf

root 457 1 0.0 Apr 29 ?? 0:14.01 /usr/local/sbin/sshd

root 484 1 0.0 Apr 29 ?? 0:10.08 /usr/bin/X11/xdm -dae

root 488 484 0.0 Apr 29 ?? 01:08:02 /usr/bin/X11/X -auth

root 493 1 0.0 Apr 29 console 0:00.13 /usr/sbin/getty conso

pndw2 16424 16509 0.0 May 14 ttyp1 0:00.29 -tcsh (tcsh)

pndw2 17138 16424 0.0 May 14 ttyp1 0:04.22 -csh (tcsh)

pndw2 30486 17138 0.0 May 23 ttyp1 5:36.69 emacs -geometry 80x30

pndw2 31804 17138 0.0 12:48:07 ttyp1 0:01.82 xdvi der.dvi

ecb34 32309 32492 0.0 20:29:07 ttyp2 0:00.31 -bash (bash)

jme23 28095 28104 0.0 May 22 ttyp4 0:00.46 -csh (csh)

Note : -1/ Note Processes not in Process ID (PID) Order.

-2/ Start Time of ’sched’ (Process ID 0) gives Boot Date.

-3/ Add ’c’ Option for Class (’CLS’), Priority (’PRI’) Data.

-4/ Add ’l’ Option for Address Space and Process Size Data.

-5/ Add ’ | grep -v " ps "’ Pipe to cut ’ps’ Command off Output.

23


*******************************

BSD UNIX (Linux, Sun OS 4.1) ’ps -aux’ Command Output :

--------------------------------------------------------

Owner Proc. %CPU %MEM SZ RSS Term. St Start CPU Command

(PID) (KB) (KB) Time Time

------ ----- ---- ---- ----- ---- ------ -- ------ ----- ---------------------

bin 193 0.0 0.5 764 352 ? S May 24 0:01 portmap

daemon 256 0.0 0.4 780 260 ? S May 24 0:00 /usr/sbin/atd

ecb34 5550 7.0 1.3 1244 872 p1 S 11:06 0:00 -bash

mbg15 473 0.0 1.1 1204 728 ? S May 24 0:00 bash /usr/damtp/defau

mbg15 479 0.0 0.3 1124 224 ? S May 24 0:00 /opt/ssh/bin/ssh-agen

mbg15 507 0.0 2.1 2252 1364 ? S May 24 0:00 xload -label inertia

mbg15 717 0.0 2.1 2360 1392 ? S May 24 0:22 xlock -allowroot -use

root 1 0.0 0.5 764 328 ? S May 24 0:04 init [5]

root 2 0.0 0.0 0 0 ? SW May 24 0:00 (kflushd)

root 3 0.0 0.0 0 0 ? SW May 24 0:00 (kpiod)

root 4 0.0 0.0 0 0 ? SW May 24 0:00 (kswapd)

root 236 0.0 0.6 824 444 ? S May 24 0:00 syslogd

root 267 0.0 0.6 852 444 ? S May 24 0:00 crond

root 278 0.0 0.6 840 424 ? S May 24 0:00 /usr/sbin/lpd -l

root 311 0.0 0.5 776 360 ? S May 24 0:00 inetd

root 371 0.0 1.5 984 984 ? S May 24 0:00 xntpd -A

root 392 0.0 0.9 1152 596 ? S May 24 0:00 rpc.nfsd

root 404 0.0 1.2 1256 820 ? S May 24 0:10 /usr/sbin/amd -F /etc

root 436 0.0 0.4 784 280 ? S May 24 0:00 gpm -t ps/2

root 443 2.2 0.7 1172 464 ? S May 24 0:02 /usr/local/sbin/sshd

root 449 0.0 1.6 2552 1068 ? S May 24 0:01 /usr/bin/X11/xdm -nod

root 453 0.3 0.8 13692 520 ? S May 24 9:10 /usr/X11R6/bin/X -aut

rrh 5485 0.0 1.5 1508 956 p0 S 09:15 0:00 -tcsh

root 7124 0.0 0.5 1496 1224 term/a S 12:50 0:00 /usr/bin/login

root 7125 0.0 0.5 1496 1224 term/b S 12:50 0:00 /usr/bin/login

root 7129 0.6 0.6 1432 1360 pts/0 O 12:54 0:00 ps -aux

Note : -1/ Note Processes not in Process ID (PID) Order.

-2/ Start Time of ’sched’ (Process ID 0) gives Boot Date.

-3/ Add ’w’ Option for 132 Char. Wide Output, with more Commands data.

-4/ Command Shown may be manipulated by Program,

Add ’c’ to get Actual Command Name held in Address Space instead.

-5/ Add ’ | grep -v " ps "’ Pipe to cut ’ps’ Command off Output.

24


*******************************

Querying current Processes :

----------------------------

To get Details, and Process ID (’PID’) of currently running Processes,

use ’ps’ Command, in combination with Options and Pipes for selectivity.

Examples : On System V, show User’s Processes with ’ps -ef | grep "^ecb34"’,

System " " ’ps -ef | grep "^root"’.

On BSD, show User’s Processes with ’ps -aux | grep "^ecb34"’,

System " " ’ps -aux | grep "^root"’.

Controlling current Processes :

-------------------------------

Users can terminate Processes, when normal Exit or Ending fails,

or request Operator to do so (Operator can terminate any Process).

To End Processes : ’kill -sss PID’, where ’sss’ = Signal Number or Mnemonic,

’PID’ = Process ID, off ’ps’ List.

Signal ’1’ = ’HUP’ = Restart,

’9’ = ’KILL’ = Stop.

’15’ = ’TERM’ = Terminate.

Examples : ’kill -HUP 449 453’ for Operator ’xdm’, ’X’ restarts in BSD List,

making them reread their Initialisation Files.

’kill -9 505 507’ for User to end 2 Processes owned by her/him.

Querying current Users on a Host :

----------------------------------

To see Users currently Logged in, use ’who’ or ’who -a’,

and to see what Commands Users are running, Enter ’w’.

25

UNIX SHELLS (COMMAND INTERPRETERS) (1)

**************************************

Role and Sorts of Shells :

--------------------------

In UNIX Command Line Sessions, all Commands Entered handled by a Shell,

or Command Interpreter, which eventually passes instructions to Kernel.

UNIX offers several Shells, of 2 Flavours : Bourne and derivatives,

’C’ " " .

Bourne Type : Bourne - oldest, smallest and most robust Shell,

(’sh’) often Default Operator and Scripting Shell.

Korn - System V Superset of Bourne Shell,

(’ksh’) with Aliases, History and Arithmetic.

Bash - Free Software Foundation Superset of Bourne,

(’bash’) also Default Linux Shell,

with Aliases, History, Arithmetic,

and emulating some ’C’ Shell Syntax.

’C’ Type : ’C’ - BSD UNIX Alternative to Bourne Shell,

(’csh’) with Aliases and History.

TCSH - Superset of, or (Linux) Substitute for ’C’.

(’tcsh’)

Common Features : Environment Variables (set for whole Log in Session),

Shell " ( " " each Shell Instance),

Built in Commands (including Logical Constructs).

Shells responsible for Expansion of : File Name Wild Cards,

Environment Variables,

Shell " ,

When available, Aliases.

When Commands not Built ins, Shell starts Sub-Shell to Execute them,

unless specifically directed not to (as in case of Read in Script).

26


**************************************

Log in and Sub Shells :

-----------------------

Users start Sessions in Shell defined with their Account Details.

Sub-Shells can be started just by Entering Shell’s Command Name,

like ’ksh’ to start a Korn Sub-Shell of current one.

Interpretation of Commands and Operands - Quoting :

---------------------------------------------------

Shell interprets Special Characters (expanding File Name Wild Cards,

or replacing Variable Names (after ’$’ Sign) by their Value) first,

that is, before actual Commands executed. To avoid this, use Quoting :

Symbol(s) Operation

--------------- --------------------------------------------------

’\’ - Backslash Escapes following Character,

which thus looses eventual special meaning.

Acts as Line Continuation Mark at End of Line,

where it escapes following Carriage Return.

’"’ - Double When text presented between Pair of Double Quotes,

Quotes Shell does not interpret Special Characters,

except Double Quotes (unless escaped with ’\’),

Single Quotes ’\’ (Backslash, Escape Character),

and ’$’ (Dollar Sign, so Variables replaced).

’’’ - Single As per Double Quotes, but without exceptions,

Quotes in effect, allowing verbatim text to pass,

to Command or Program.

For other special Characters, see notes about UNIX Files Names (1).

27


**************************************

Variables :

-----------

Exist at 2 Levels : Environment - Apply across a whole Session,

that is to all Shell and Program

Instances being run under it.

Shell - Apply only to a Shell Instance.

Set in 2 ways : Automatic - May be Session Defaults,

Command execution Parameters,

or Command Outcome Flags.

Session - Non Default Variables may be set,

and Session Defaults altered,

either on Command Line,

or via Filed Commands (a Script).

Variable setting syntax differs between Bourne and ’C’ Type Shells :

Bourne Type : ’export NAME="value"’, or (Bourne),

’NAME="value"’, and ’export NAME’.

’C’ Type : ’setenv NAME "value"’ (Env.),

’set NAME = ( "value" )’ (Shell).

In both cases Var. Name given without ’$’ Prefix,

Quotes (Double or Single) Optional.

To show Variable : Enter ’echo $VAR_NAME’ , to get ’value’,

or ’echo "$VAR_NAME"’, " " ’"value"’.

To show all Vars. : Environment Variables, Enter ’env’,

Shell " , " ’set’.

28


**************************************

Some Common Session Environment Variables :

-------------------------------------------

Name Description

------------ ----------------------------------------------------------

’HOME’ Contains Home Directory of Current User,

so, ’cd $HOME’ moves Session to that Home Dir.

All Shells apart Bourne also support ’~’ (Tilde).

’PATH’ Set of Directories searched for finding Commands.

Allows Commands to be entered without Directory prefixed.

Successive Directories in Path separated by Colon Symbols,

like ’/bin:/usr/bin:/usr/local/bin’. Others can be added.

Current and Higher Dir. ’.’ and ’..’ shorthands valid,

as are Variables (so ’/bin:$HOME/bin:.:./bin’ correct).

’PATH’ may be altered mid Session. When ’$PATH’ in Value,

Current Value preserved, with extra Dirs. added or tailed.

’prompt’ Terminal (Command) Prompt for ’C’ Type Shells,

assigned with Command like ’set prompt = ( "> " )’.

’prompt2’ Command Continuation Prompt for ’C’ Type Shells.

’PS1’ Terminal (Command) Prompt for Bourne Type Shells.

assigned with Command like ’export PS1="$ "’.

’PS2’ Command Continuation Prompt for Bourne Type Shells.

’TERM’ Terminal (emulation) Type. Needed by some Editors.

Try "vt220" (an old DEC VDU Model) if unsure what to use.

’LOGNAME’ User Name for Current Session (also ’USER’).

’EDITOR’ Optional Default Text Editor (used by some Mail Clients),

defines Command Line Editing operation in Korn and Bash.

’PRINTER’ Optional Default Printer (used by Mail or other Programs).

’SHELL’ Should contain Path and Name of Log in Shell,

as given in User’s Entry in Password File ’/etc/passwd’.

29


**************************************

Command Parameters and Command Outcome Variables :

--------------------------------------------------

Commonly labelled with prefixed ’$’ (Dollar) Sign,

unlike Environment or Session and other Variables,

so, for example "DollarDollar" stands for ’$$’.

These prove useful mostly in Scripts, to control execution,

depending on outcome of one or more prior operations,

but some "Dollar" Vars. can also help in Interactive Sessions.

$Name Description

----- ----------------------------------------------------------

’$$’ Process Number of Current Command (or Script),

as would show in Output of ’ps -aux’ or ’ps -ef’.

’$!’ Process Number of last (when any) Background Job,

that is Command submitted with ’&’ (Ampersand) appended,

where Session no longer waits for Outcome to prompt again.

’$?’ Numerical Value representing Outcome of last Command.

Often called "Return Code",

and explained in ’man’ Page for that Command.

’$0’ Name of Current Command or Script.

’$-’ Options in effect with Current Command.

’$n’ With ’n’ a Number from ’1’ to possibly ’9’,

or more under Bash and Korn, when written ’$[nn]’,

represents Command Line Argument in that Position,

so for example, with ’ls -al’, ’$1’ Value ’-al’.

More meaningful in Scripts.

’$*’ String of all Command Line Arguments.


’$#’ Total Number of Command Line Arguments.


30


**************************************

Aliases :

---------

User defined Pseudonyms for Commands or Scripts,

possibly including any number of Command Arguments.

Supported by all Shells except Bourne,

but with different syntax between Shell Families.

Once set, Aliases may be submitted instead of Command,

and any Arguments included when Alias defined.

An Alias (say ’aname’) may be unset with ’unalias aname’.

Aliases only known in Shell Instance where defined.

Setting Aliases :

-----------------

Korn or Bash Shells : ’alias aname="cname"’,

where ’aname’ Alias Name,

and ’cname’ Command and (any) Arguments,

which must be Quoted when Spaces present.

For example ’alias ltmp="ls -al /tmp/"’,

sets an Alias called ’ltmp’ to list ’/tmp/’.

’C’ Type Shells : ’alias aname "cname"’,

where ’aname’ and ’cname’ as above for Bash.

For example ’alias ltmp="ls -al /tmp/"’,

sets an Alias called ’ltmp’ to list ’/tmp/’.

For all Shells : ’unalias aname’ drops Alias ’aname’.

Alias Names : Must start with an Alphanumeric Character,

and match Command Names, but that unadvised.

Also not necessarily wise to set many Aliases,

as this can confuse.

31


**************************************

Command History :

-----------------

Facility to retrieve, Edit and resubmit previous Commands.

These known under absolute as well as relative Numbers,

in History File (shown with Command ’history’).

Supported by all Shells except Bourne,

but with different operation between Shell Families.

History of Commands only known in current Shell Instance.

Bash History handling :

-----------------------

Arrow Keys : Up and Down Keys bring Previous or Next History Entry.

Left and Right Keys place Cursor along Line shown.

Control Key : With ’p’, ’n’, same effect as Up, Down Arrow Keys.

With ’b’, ’f’, same effect as Left, Right Arrow Keys.

With ’a’, ’e’, move to Start or End of current Line.

With ’d’, deletes current Character,

’k’, deletes all Characters from current,

’i’, inserts typed Characters at current Position.

Backspace : Deletes Character to left of Current.

Delete Key : Deletes current Character.

!! : Resubmits Last Command.

!-n : Resubmits nth Previous Command in History.

!n : Resubmits Command Number ’n’ in History File,

contents of which and Numbers shown with ’history’.

!cnmame : Resubmits last Command beginning with ’cname’.

32


**************************************

Korn History handling :

-----------------------

Set Environment Var. "EDITOR" to "emacs" (’export EDITOR="emacs"),

to enable Control Key operation similar to Bash Shell above.

Korn Shell uses ’r ’ (retrieve, with Space appended),

to replicate Exclamation Mark (’!’) operation of Bash above,

except ’r’ alone needed for repeat of Previous Command.

So, for example, ’r -4’ would rerun Command 4 up in History,

or, ’r ls’ would rerun last ’ls’ with Options and Arguments.

Arrow Keys not set for History handling in standard Korn Shell.

’C’ Shells Family History handling :

------------------------------------

’C’ Shell uses Exclamation Mark (’!’) operation as Bash above.

Additionally, ’tcsh’ uses Arrow Key operation as in Bash above.

33

UNIX SHELL SCRIPTS (1)

**********************

Scripts are sets of Shell inputs interpreted, one line at a time,

from Files rather than an online Session. Any sequence of Interactive

Inputs can be written into a Script, for any Shell on any UNIX System.

Scripts can either be Read as Commands (as if typed) for Current Shell,

in which case Files holding them need only Read (’r’) Access for User,

or Executed as Commands, via a transient Sub-Shell of Current Shell,

Script Files then needing Read (’r’) and Execute (’x’) Permissions.

A Read in Script must be syntactically compatible with Current Shell,

but Executed Scripts may use a Shell different from that invoking them.

To Read Script : ’. script_path/script_file’ for : Bourne (’sh’ ),

Korn (’ksh’ ),

Bash (’bash’).

’source path/script_file’ for : ’C’ (’csh’ ),

tcsh (’tcsh’),

Bash (’bash’).

To Execute Script : Enter optional Path and Script Name as any Command.

First Line of Script File should begin with ’#!/Dir/Shell’,

where ’/dir/Shell’ gives full Path to desired Shell Binary Module.

When not, Script can only use a Shell compatible with calling Session.

Scripts may call other Scripts, either Reading them, or Executing them,

and same Shell compatibility restrictions apply to such called Scripts,

as with Scripts invoked on Command Line with respect to Session Shell.

34

UNIX SHELL SCRIPTS (2)

**********************

Which Shell to use :

--------------------

Bourne Shell (’/bin/sh’) : Present on any UNIX System,

Compact,

Robust,

Upward compatible with Korn and Bash,

i.e. these can Read Bourne Scripts.

Korn Shell (’/bin/ksh’) : Robust and more functionality than Bourne,

Present on System V UNIX implementations

(AIX, Solaris 2.x, DEC, HP-UX),

Public Domain Version available for Linux.

’C’ Shell (’/bin/csh’) : Not consistent with Interactive behaviour.

Lacks many useful Functions for Scripts,

Many bugs in several implementations.

Bash and ’tcsh’ : Optional (except on Linux),

More Robust and consistent than ’C’ Shell,

Bash combines Korn and ’C’ functionality.

In general, if possible, Bourne Shell best, and traditionally advised.

Most extended functionality found with other Shells achievable in it,

at cost of longer Scripts, or more use of Operating System Commands,

as in case of arithmetical work.

Bash Shell, Default on Linux, offers several useful extra features,

but should be used as a Bourne Shell Superset for Scripting.

35

DEVELOPING UNIX SHELL SCRIPTS (1)

*********************************

Can start with Interactive submission of Commands. These can be

filed at same time with ’script File_name’, till ’exit’ Entered,

to end filing of Commands. However, ’script’ also saves Output,

as well as Commands, effectively filing complete Session. So,

Commands wanted to build Script may be mixed with many other Lines.

In Shells supporting Command History (all except Bourne), History

File or ’history’ Command Output may form base for Script. This

avoids problem of clearing Command Output Lines to build Script.

Initial extract can then be edited with a Text Editor.

When Session run in a Terminal Window on a Graphical Display,

previous Commands, may be cut and pasted, or some ’history’ Output.

Scripts can also be created for complex one off needs,

using Output Redirection and Pipes into and from Stream Editors,

like ’grep’, ’sed’ and ’awk’. ’history’ Output may be used thus :

history | tail -12 | head -11 | \

sed ’s/^ [0-9][0-9]*./ /1’ > TstScrpt

Above takes last 11 Lines from ’history’ (without Current Command,

which will show as Last Line of ’history’ File), cuts out Leading

Command Numbers, and writes Script File ’TstScrpt’.

In Bourne Type Sells, Functions can be written for Current Shell,

or inside a Script for its own use. Scripts can call others,

as Sub-Scripts, just as current Shell runs them, or passes control,

via ’exec’ Command (use with caution as errors can crash Session).

36


*********************************

Current Shell "On the hoof" Script Examples :

---------------------------------------------

Script made to become one Shell Command, from ’(’ to ’)’,

using internal ’;’ Separators, to show top 3 Lines of a File,

as would ’head -3’, using Input Redirection (’<’) to pick File.

Note, Continuation Prompts ’>’ (’$PS2’ Default) after ’\’ Escapes :

$ ( read LINE ; echo "$LINE" ; \

> read LINE ; echo "$LINE" ; \

> read LINE ; echo "$LINE" ) < path_name/file_name

Function ’shcat’ to Simulate ’cat’ Command. Note,

Continuation Prompts ’>’ (’$PS2’ Default) till closing ’}’ :

$ shcat ( ) {

>

> #### TEST FOR "$n" ARGUMENTS, I.E. FILE NAMES.

>

> while (test "$#" -ge "1")

> do

>

> #### PICK NEXT LINE OFF INPUT FILE,

> #### AND DISPLAY IT TO DEFAULT OUTPUT.

>

> while read LINE

> do

>

> echo "$LINE"

>

> done < "$1"

>

> #### PICK NEXT FILENAME IN ARGUMENT LIST.

>

> shift

>

> done }

$

37


*********************************

Building One Off Script to Copy some Files to a transfer Area,

and cut out redundant Middle Name part, so Names 8 Chars. at most,

for possible copy to a Microsoft DOS Personal Computer.

$ #### CREATE INPUT FILE LIST.

$

$ ls /export/home/Oper/wrk/ima/*.ima > ./list.001

$

$ cat list.001

/export/home/Oper/wrk/ima/2588-1-1.ima




$

$ #### CREATE OUTPUT FILE LIST,

$ #### REPLACING DIRECTORY (PATH) (BETWEEN FIRST AND LAST ’/’).

$

$ cat list.001 | sed ’s%/.*/%/tmp/cop/%g’ > list.002

$

$ cat list.002

/tmp/cop/2588-1-1.ima




$

$ #### CUT OUT OUTPUT FILE NAMES MIDDLE FIELD (BETWEEN ’-’).

$

$ cat list.002 | sed ’s/-.*-/-/g’ > list.003

$

$ cat list.003

/tmp/cop/2588-1.ima

/tmp/cop/3529-1.ima

/tmp/cop/3562-15.ima

/tmp/cop/3562-16.ima

$

... continued ...

38


*********************************

One off Script, ... continued ...

$ #### MERGE RESPECTIVE INPUT AND OUTPUT FILE NAMES.

$

$ paste -d" " list.001 list.003 > list.004

$

$ cat list.004

/export/home/Oper/wrk/ima/2588-1-1.ima /tmp/cop/2588-1.ima




$

$ #### PREPEND ’cp’ COMMAND TO CREATE ONE OFF SCRIPT.

$

$ cat list.004 | sed ’s/^/cp /g’ > Frename

$

$ cat Frename

cp /export/home/Oper/wrk/ima/2588-1-1.ima /tmp/cop/2588-1.ima




$

$ #### MAKE SHELL READ AND EXECUTE COMMANDS IN ONE OFF SCRIPT.

$

$ . ./Frename

$

$ #### CHECK RESULTS OF RUNNING SCRIPT. USE FEW FILES INITIALLY.

$

$ ls -al /tmp/cop

total 1616

drwxr-sr-x 2 oper daemon 199 Nov 5 19:01 .

drwxrwsrwt 6 daemon daemon 524 Nov 5 19:01 ..

-r--r--r-- 1 oper daemon 137216 Nov 5 19:01 2588-1.ima




$

39


*********************************

Generic Script to write, run and delete equivalent to above one off.

#!/bin/sh

#### SCRIPT FOR COPY OF IMAGE FILES TO TRANSFER AREA

#### ***********************************************

#### NOTE TOP LINE FORCES ’BOURNE’ SHELL FOR REST OF SCRIPT.

#### SCRIPT COPIES IMAGE FILES TO SPECIAL TRANSFER DIRECTORY,

#### CUTTING OUT MIDDLE (STUDY NUMBER) AS REDUNDANT,

#### SO AS TO FIT NAME IN 8 CHAR. AND 3 CHAR. EXTENSION,

#### FOR POSSIBLE ’ftp’ TRANSFERS TO SYSTEMS WITH THIS LIMIT.

#### OUTPUT FEED BACK HEADING.

echo ""

echo "IMAGE (OR RAW) FILE COPY FOR ’ftp’ TRANSFER"

echo "*******************************************"

#### SOURCE IMAGE FILE(S) DIRECTORY, AND SUBDIRECTORY,

#### FOR RENAMED FILE(S) COPY AND TRANSIENT SCRIPTS.

IDIR="/disk2"

TDIR="$IDIR/cop"

#### WHEN NOT GIVEN AS CALL ARGUMENT,

#### REQUEST PATIENT SYSTEM NUMBER ENTRY, FOR ALL IMAGES,

#### PATIENT-STUDY NUMBERS FOR ONLY THAT STUDY,

#### OR PATIENT-STUDY-IMAGE FOR SET IMAGE FILE.

if (test ".$1" != ".")

then

PNUM="$1"

fi

... continued ...

40


*********************************

Generic Script, continued (1) ...

#### UNTIL VALID SELECTION MADE (AT LEAD ONE IMAGE FILE FOUND),

#### CYCLE THROUGH ENTRY, CHECK MESSAGE STATEMENTS,

#### NOTING MENU NOT SHOWN WHEN INITIAL PASSED SELECTION ARGUMENT.

while (test ".$PEXS" = ".")

do

if (test ".$PNUM" = ".")

then

echo ""

echo "IMAGE FILES COPY TO ’ftp’ TRANSFER AREA :"

echo "------------------------------------------"

echo "Enter : Pat. System No. for All Images,"

echo " Pat.-Study ’’ ’’ Study ’’ ,"

echo " Pat.-Study-Image ’’ single Image,"

echo " ’0’ to Exit ."

read PNUM

fi

if (test ".$PNUM" = ".0")

then

exit

fi

PEXS=‘ls "$IDIR"/"$PNUM"*.??? 2>/dev/null | \

head -1 ‘

if (test ".$PEXS" = ".")

then

echo ""

echo "**** No $PNUM Images or Raw Files ****"

PNUM=""

fi

done

continued ...

41


*********************************


#### SPLIT ENTERED FILE SELECTION NUMBER(S) INTO PART,

#### FOR PATIENT, STUDY AND IMAGE NUMBER(S),

#### USING ’-s’ OPTION TO AVOID OUTPUT WHEN NO NUMBER.

FNU1=‘echo "$PNUM" | \

cut -d"-" -f1 ‘


cut -d"-" -f2 -s ‘


cut -d"-" -f3 -s ‘

#### STORE APPROPRIATE FILE LIST TO COPY AND RENAME,

#### DEPENDING ON WHETHER ALL PATIENT, ONE STUDY OR ONE IMAGE NEEDED.

if (test ".$FNU3" != ".")

then

ls "$IDIR"/"$PNUM".??? > "$TDIR"/ftpc"$PNUM".s1

echo ""

echo "Copy File ’$PNUM.???’ to ’$TDIR’"

else

if (test ".$FNU2" != ".")

then

ls "$IDIR"/"$PNUM"-*.??? > "$TDIR"/ftpc"$PNUM".s1

echo ""

echo "Copy Files ’$PNUM-*.???’ to ’$TDIR’"

else

ls "$IDIR"/"$PNUM"-*-*.??? > "$TDIR"/ftpc"$PNUM".s1

echo ""

echo "Copy Files ’$PNUM-*-*.???’ to ’$TDIR’"

fi

fi

... continued ...

42


*********************************


#### CREATE STREAM EDITOR ’sed’ SCRIPT FOR FILING,

#### TO CUT OUT DIRECTORY NAME FROM FILE LIST TO COPY,

#### DIR. NAME BEING CHARS FROM LEADING ’/’ TO LAST ’/’ INCLUSIVE,

#### AND REPLACE IT BY DESTINATION DIR. NAME (OFF ENV. VARIABLE).

#### SCRIPT USING ’%’ ALTERNATE SEPARATORS TO HANDLE ’/’ IN NAMES,

#### AND NEEDS SUBMISSION OFF FILE VIA ’-f’ OPTION TO ’sed’,

#### AS ENV. VARIABLE PREVENTS USING SINGLE QUOTES AROUND SCRIPT.

echo "s%^/.*/%$TDIR/%g" > "$TDIR"/ftpc"$PNUM".ss

#### FEED INPUT FILE NAMES WITH DIR. PREFIX TO STREAM EDITOR ’sed’,

#### TO REPLACE SOURCE DIR. WITH DESTINATION IN OUTPUT FILES LIST,

#### AS PER SUBSTITUTE (’s’) COMMAND IN FILED ’sed’ SCRIPT.

#### THEN CUT OUT MIDDLE INPUT NAME FIELD IN OUTPUT FILE NAMES,

#### CHANGING CENTRAL ’-digits-’ TO ’-’ WITH ’sed’ STREAM EDITOR.

cat "$TDIR"/ftpc"$PNUM".s1 | \

sed -f "$TDIR"/ftpc"$PNUM".ss | \

sed ’s/-[0-9]*-/-/1’ > "$TDIR"/ftpc"$PNUM".s2

#### MERGE SOURCE-DESTINATION FILE NAME LISTS,

#### THEN PREPEND ’cp ’ COPY COMMAND TO ALL LINES.

paste -d" " "$TDIR"/ftpc"$PNUM".s1 \

"$TDIR"/ftpc"$PNUM".s2 | \

sed ’s/^/\/bin\/cp /g’ > "$TDIR"/ftpc"$PNUM".s3

#### EXECUTE OUTPUT FILE AS BOURNE SHELL SCRIPT,

#### THEN DELETE IT AND FILE LISTS.

. "$TDIR"/ftpc"$PNUM".s3

rm "$TDIR"/ftpc"$PNUM".s?

#### OUTPUT END OF RUN MESSAGE.

echo ""

echo "**** Copy successful ****"

43


*********************************

Generic Script, Notes :

-----------------------

First ensure Script generates correct one off Script,

by Commenting out one off Script Read (’.’) Line,

and Printing Script with a ’cat’ Command instead :



echo ""

cat "$TDIR"/ftpc"$PNUM".s3

# . "$TDIR"/ftpc"$PNUM".s3


Execute Generic Script to Check out Script Output :

$ /usr/appl/adm/scr/ftpcop 2534-1

IMAGE (OR RAW) FILE COPY FOR ’ftp’ TRANSFER

*******************************************

Copy Files ’2534-1-*.???’ to ’/disk2/cop’

cp /disk2/2534-1-1.ima /disk2/cop/2534-1.ima





**** Copy successful ****

$

44


**********************************

Finalising Generic Script :

---------------------------

Once test without copy of Files successful,

set up definitive form of Generic Script :



echo ""

. "$TDIR"/ftpc"$PNUM".s3


Execute Script on some (small set of) Test Files.

$ /usr/appl/adm/scr/ftpcop 2534-1

IMAGE (OR RAW) FILE COPY FOR ’ftp’ TRANSFER

*******************************************

Copy Files ’2534-1-*.???’ to ’/disk2/cop’

**** Copy successful ****

$

Check outcome as intended.

$ ls -al /disk2/cop

total 946

drwxrwx--- 2 mr 512 Nov 5 20:18 .

drwxr-x--- 4 mr 214016 Nov 5 20:08 ..

-rw-rw-r-- 1 work 137216 Nov 5 20:18 2534-1.ima





$

45

UNIX, THE LITERATURE

********************

The first ten or so years of UNIX development saw few books published

on the subject. Indeed, the official Manuals themselves were quite

sparse. A lot of knowledge was passed informally between researchers,

feeding the inscrutable reputation of the Operating System.

This changed from the mid eighties onward. An increasing number of

titles appeared, catering to all levels of expertise. An explosion then

followed in the last few years, with the similar growth of the Internet.

Even when considering yourselves absolute novices, you can ignore the

’beginner’ level books and pick the more useful ’for all users’ sort.

Remember also that the Command ’man cccc’ will show a description of the

actual Command ’cccc’, together with a list of its options, attributes

and related Commands. It is accurate for the Host Computer, and will

even list any bugs. UNIX pioneered on line documentation. It felt less

of a chore for the early developers than keeping up to date Manuals,

partly compensating for their paucity.

The O’Reilly publications are my favourite, competent but easy to read

and keenly priced, unlike some others. The Mc Graw Hill ’Osborne’ set

also offers good references, especially for general applications use.

Here follows a selection of titles I have myself used or been advised.

46

Some UNIX (Book) Titles (1)

===========================

Historical view :

-----------------

P. Libes, S. Ressler,

Life with UNIX,

Prentice Hall, 1992.

General Introductions/descriptions :

------------------------------------

S. Coffin,

UNIX System V Release 4, the Complete Reference,

Osborne Mc Graw Hill, ISBN 0-07-881653-X, 1991.

D.S. Ray, E.J. Ray,

UNIX (Visual Quickstart Guide),

Peachpit Press (Addison Wesley), ISBN 0-201-35395-4, 1998.

K.H. Rosen, R.R. Rosinsky, J.M. Fraser,

UNIX System V Release 4, An Introduction for New and Experienced Users,

Osborne Mc Graw Hill, ISBN 0-07-881552-5, 1990.

’X’ Windows Introduction and Library References :

-------------------------------------------------

E. Cutler, D. Gilly, T. O’Reilly

The X Window System in a Nutshell.

O’Reilly & Associates, ISBN 1-56592-017-1, 1992.

N. Mansfield

The Joy of X, an Overall View of the X Window System.

Addison Wesley, ISBN 0-221-65612-9, 1992.

47


===========================

Command and Syntax References :

-------------------------------

P. DuBois,

Using csh & tcsh,


D. Gilly,

UNIX in a Nutshell (3nd ed.),


L. Lamb,

Learning the Vi Editor (6th ed.),


B. McCarty,

Learning Red Hat Enterprise Linux & Fedora (4th ed.),

O’Reilly & Associates, ISBN 0-596-00589-X, 2004.

C. Newham, B. Rosenblatt,

Learning the Bash Shell (2nd ed.),


B. Rosenblatt, A. Robbins

Learning the Korn Shell (2nd ed.),


E. Siever, S. Figgins, A. Weber,

Linux in a Nutshell (4th ed.),

O’Reilly & Associates, ISBN 596-00482-6, 2003.

48


===========================

For more technical Users :

--------------------------

D.A. Curry,

UNIX Systems Programming for SVR4,


A. Frisch,

Essential System Administration (2nd ed.),


G. Lehey,

Porting UNIX Software,


L. Mui, E. Pearce,

X, Volume 8: X Window System Administrator Guide,


L. Mui, V. Quercia,

X User Tools,


E. Nemeth, G. Snyder, T.R. Hein,

Linux Administration Handbook,

Prentice Hall, ISBN 0-13-008466-2, 2002.

E. Nemeth, G. Snyder, S. Seebass, T.R. Hein,

UNIX System Administration Handbook (3rd ed.),

Prentice Hall, ISBN 0-13-020601-6, 2001.

J. Peek, T. O’reilly, M. Loukides,

UNIX Power Tools (3rd ed.),


M. Welsh, M. Kalle Dalheimer, L. Knaufman,

Running Linux (4th ed.),


49

introduction - defining · pdf fileintroduction - defining unix ... built on free softwre...

Documents