Download - Chapter 3 UNIX Utilities for Power Users
![Page 1: Chapter 3 UNIX Utilities for Power Users](https://reader034.vdocument.in/reader034/viewer/2022051402/568158be550346895dc6090d/html5/thumbnails/1.jpg)
Chapter 3UNIX Utilities for Power Users
Graham Glass and King Ables, UNIX for Programmers and Users,
Third Edition, Pearson Prentice Hall, 2003.
Original Notes by Raj SunderramanConverted to presentation and updated by
Michael Weeks
![Page 2: Chapter 3 UNIX Utilities for Power Users](https://reader034.vdocument.in/reader034/viewer/2022051402/568158be550346895dc6090d/html5/thumbnails/2.jpg)
Grep(Global or Get Regular Expression and Print)
Filtering patterns: egrep, fgrep, grep grep -hilnvw pattern {fileName}*
displays lines from files that match the pattern
pattern : regular expression -h : do not list file names if many files are specified -i : ignore case -l : displays list of files containing pattern -n : display line numbers -v : displays lines that do not match the pattern -w : matches only whole words only
![Page 3: Chapter 3 UNIX Utilities for Power Users](https://reader034.vdocument.in/reader034/viewer/2022051402/568158be550346895dc6090d/html5/thumbnails/3.jpg)
Grep variations
fgrep : pattern must be fixed string
egrep : pattern can be extended regular expression -x option in fgrep: displays only lines that are exactly
equal to string
extended regular expressions: + matches one or more of the single preceding character ? matches zero or one of the single preceding character | either or (ex. a* | b*) () *, +, ? operate on entire subexpression not just on
preceding character; ex. (ab | ba)*
![Page 4: Chapter 3 UNIX Utilities for Power Users](https://reader034.vdocument.in/reader034/viewer/2022051402/568158be550346895dc6090d/html5/thumbnails/4.jpg)
Differences
gerp Search a Pattern from current directory.
egrep (grep -E in linux) is extended grep where additional regular expression metacharacters have been added like +, ?, | and ().
fgrep (grep -F in linux) is fixed or fast grep and behaves as grep but does not recognize any regular expression metacharacters as being special.
![Page 5: Chapter 3 UNIX Utilities for Power Users](https://reader034.vdocument.in/reader034/viewer/2022051402/568158be550346895dc6090d/html5/thumbnails/5.jpg)
uniq
fed with a text file, output the file with adjacent identical lines collapsed to one
uniq -c -number [inputfile [outputfile]]
-c: causes each line to be preceded by the number of occurrences that were found
-number: number fields of each line are ignored
![Page 6: Chapter 3 UNIX Utilities for Power Users](https://reader034.vdocument.in/reader034/viewer/2022051402/568158be550346895dc6090d/html5/thumbnails/6.jpg)
Sort
sorts a file in ascending or descending order based on one or more fields.
Individual fields are ordered lexicographically, which means that corresponding characters are compared based on their ASCII value.
![Page 7: Chapter 3 UNIX Utilities for Power Users](https://reader034.vdocument.in/reader034/viewer/2022051402/568158be550346895dc6090d/html5/thumbnails/7.jpg)
Sort
sort -tc -r [+POS1 [-POS2]] {sortField -bfMn}* {fileName}* -tc separator is c instead of blank
-r descending instead of ascending
-b ignore leading blanks
-f ignore case
-M month sort (3 letter month abbreviation)
-n numeric sort
+POS1 [-POS2] key positions start [up to end]
![Page 8: Chapter 3 UNIX Utilities for Power Users](https://reader034.vdocument.in/reader034/viewer/2022051402/568158be550346895dc6090d/html5/thumbnails/8.jpg)
Sort Examples
$ cat sort.dat John Smith 1222 20 Apr 1956 Tony Jones 1012 20 Mar 1950 John Duncan 1111 20 Jan 1966 Larry Jones 1223 20 Dec 1946
$ sort +0 -2 sort.dat John Duncan 1111 20 Jan 1966 John Smith 1222 20 Apr 1956 Larry Jones 1223 20 Dec 1946 Tony Jones 1012 20 Mar 1950
![Page 9: Chapter 3 UNIX Utilities for Power Users](https://reader034.vdocument.in/reader034/viewer/2022051402/568158be550346895dc6090d/html5/thumbnails/9.jpg)
Sort Examples
$ sort +4 -5 -M sort.dat John Duncan 1111 20 Jan 1966 Tony Jones 1012 20 Mar 1950 John Smith 1222 20 Apr 1956 Larry Jones 1223 20 Dec 1946
$ sort +4 -5 sort.dat John Smith 1222 20 Apr 1956 Larry Jones 1223 20 Dec 1946 John Duncan 1111 20 Jan 1966 Tony Jones 1012 20 Mar 1950
![Page 10: Chapter 3 UNIX Utilities for Power Users](https://reader034.vdocument.in/reader034/viewer/2022051402/568158be550346895dc6090d/html5/thumbnails/10.jpg)
Comparing Files
What if you have two files with the same name, but different sizes?
cmp -ls file1 file2 offset1 offset2 compares two files for equality
reports the first byte where there is a mismatch
if one file is a prefix of the other, EOF message is displayed
![Page 11: Chapter 3 UNIX Utilities for Power Users](https://reader034.vdocument.in/reader034/viewer/2022051402/568158be550346895dc6090d/html5/thumbnails/11.jpg)
Comparing Files
cmp -ls file1 file2 offset1 offset2
Optional values offset1 and offset2 are the offsets into the files where comparison begins. -l option displays the line number and byte offset of
all mismatched bytes
-s option suppresses all output
![Page 12: Chapter 3 UNIX Utilities for Power Users](https://reader034.vdocument.in/reader034/viewer/2022051402/568158be550346895dc6090d/html5/thumbnails/12.jpg)
Comparing Files
diff -i file1 file2 compares two files
outputs a description of their differences
-i flag ignores case
![Page 13: Chapter 3 UNIX Utilities for Power Users](https://reader034.vdocument.in/reader034/viewer/2022051402/568158be550346895dc6090d/html5/thumbnails/13.jpg)
Archiving
tar -cfrtuvx tarFileName fileList creates a “tape archive”-format file from the fileList
Typically, use either -c or -x, but not both! -c for Create -x for Extracts
-t option generates a table of contents
-r option unconditionally appends listed files to tar formatted file
![Page 14: Chapter 3 UNIX Utilities for Power Users](https://reader034.vdocument.in/reader034/viewer/2022051402/568158be550346895dc6090d/html5/thumbnails/14.jpg)
Archiving tar -cfrtuvx tarFileName fileList
-u option appends only files that are more recent than those already archived
-f option enables you to give a tar file name Default name is /dev/rmt0
-v verbose
If fileList contains directory, its contents are appended/extracted recursively.
![Page 15: Chapter 3 UNIX Utilities for Power Users](https://reader034.vdocument.in/reader034/viewer/2022051402/568158be550346895dc6090d/html5/thumbnails/15.jpg)
Archive Example
$ tar cvf ch6.tar ch6 ch6/ ch6/menu.csh ch6/junk/ ch6/junk/junk.csh ch6/junk.csh ch6/menu2.csh ch6/multi.csh ch6/expr1.csh ch6/expr3.csh ch6/expr4.csh ch6/if.csh ch6/menu3.csh
![Page 16: Chapter 3 UNIX Utilities for Power Users](https://reader034.vdocument.in/reader034/viewer/2022051402/568158be550346895dc6090d/html5/thumbnails/16.jpg)
Archive Example
$ ls -l ch6.tar -rw-rw-r-- 1 raj raj 20480 Jun 26 20:08 ch6.tar
$ tar -tvf ch6.tar drwxr-xr-x raj/raj 0 2007-06-03 09:57 ch6/ -rwxr-xr-x raj/raj 403 2007-06-02 14:50 ch6/menu.csh drwxr-xr-x raj/raj 0 2007-06-03 09:57 ch6/junk/ -rwxr-xr-x raj/raj 1475 2007-06-03 09:57 ch6/junk/junk.csh -rwxr-xr-x raj/raj 1475 2007-06-03 09:56 ch6/junk.csh -rw-r--r-- raj/raj 744 2007-06-02 15:59 ch6/menu2.csh -rwxr-xr-x raj/raj 445 2007-06-02 15:26 ch6/multi.csh -rwxr-xr-x raj/raj 279 2007-06-02 15:18 ch6/expr1.csh -rwxr-xr-x raj/raj 98 2007-06-02 15:20 ch6/expr3.csh -rwxr-xr-x raj/raj 262 2007-06-02 15:21 ch6/expr4.csh -rwxr-xr-x raj/raj 204 2007-06-02 15:22 ch6/if.csh -rw-r--r-- raj/raj 744 2007-06-02 16:01 ch6/menu3.csh
![Page 17: Chapter 3 UNIX Utilities for Power Users](https://reader034.vdocument.in/reader034/viewer/2022051402/568158be550346895dc6090d/html5/thumbnails/17.jpg)
Archive Example $ rm -fr ch6
$ tar -rvf ch6.tar date.txt date.txt
$ tar -tvf ch6.tar drwxr-xr-x raj/raj 0 2007-06-03 09:57 ch6/ -rwxr-xr-x raj/raj 403 2007-06-02 14:50 ch6/menu.csh drwxr-xr-x raj/raj 0 2007-06-03 09:57 ch6/junk/ -rwxr-xr-x raj/raj 1475 2007-06-03 09:57 ch6/junk/junk.csh -rwxr-xr-x raj/raj 1475 2007-06-03 09:56 ch6/junk.csh -rw-r--r-- raj/raj 744 2007-06-02 15:59 ch6/menu2.csh -rwxr-xr-x raj/raj 445 2007-06-02 15:26 ch6/multi.csh -rwxr-xr-x raj/raj 279 2007-06-02 15:18 ch6/expr1.csh -rwxr-xr-x raj/raj 98 2007-06-02 15:20 ch6/expr3.csh -rwxr-xr-x raj/raj 262 2007-06-02 15:21 ch6/expr4.csh -rwxr-xr-x raj/raj 204 2007-06-02 15:22 ch6/if.csh -rw-r--r-- raj/raj 744 2007-06-02 16:01 ch6/menu3.csh -rw-rw-r-- raj/raj 29 2007-06-21 11:06 date.txt
![Page 18: Chapter 3 UNIX Utilities for Power Users](https://reader034.vdocument.in/reader034/viewer/2022051402/568158be550346895dc6090d/html5/thumbnails/18.jpg)
Archive Example
$ tar -xvf ch6.tar ch6/ ch6/menu.csh ch6/junk/ ch6/junk/junk.csh ch6/junk.csh ch6/menu2.csh ch6/multi.csh ch6/expr1.csh ch6/expr3.csh ch6/expr4.csh ch6/if.csh ch6/menu3.csh date.txt
![Page 19: Chapter 3 UNIX Utilities for Power Users](https://reader034.vdocument.in/reader034/viewer/2022051402/568158be550346895dc6090d/html5/thumbnails/19.jpg)
Find Utility
find pathList expression
finds files starting at pathList
finds files descending from there
Allows you to perform certain actions e.g. deleting the files
![Page 20: Chapter 3 UNIX Utilities for Power Users](https://reader034.vdocument.in/reader034/viewer/2022051402/568158be550346895dc6090d/html5/thumbnails/20.jpg)
Find Utility name pattern
true if the file name matches pattern
perm oct
true if the octal description of file's permission equals oct
type ch
true if the type of the file is ch (b=block, c=char ..)
user userId
true if the owner of the file is userId
group groupId
true if the group of the file is groupId
![Page 21: Chapter 3 UNIX Utilities for Power Users](https://reader034.vdocument.in/reader034/viewer/2022051402/568158be550346895dc6090d/html5/thumbnails/21.jpg)
Find Utility
-atime count
true if the file has been accessed within count days
-ctime count
true if the contents of the file have been modified within count days or any of its file attributes have been modified
-exec command
true if the exit code = 0 from executing the command.
command must be terminated by \; If {} is specified as a command line argument it is
replaced by the file name currently matched
![Page 22: Chapter 3 UNIX Utilities for Power Users](https://reader034.vdocument.in/reader034/viewer/2022051402/568158be550346895dc6090d/html5/thumbnails/22.jpg)
Find Utility
prints out the name of the current file and returns true
-ls
displays the current file's attributes and returns true
!expression
negation of expression
expr1 [-a] expr2
short circuit and
expr1 -o expr2
short circuit or
![Page 23: Chapter 3 UNIX Utilities for Power Users](https://reader034.vdocument.in/reader034/viewer/2022051402/568158be550346895dc6090d/html5/thumbnails/23.jpg)
Find Examples
$ find / -name x.c searches for file x.c in the entire file system
$ find . -mtime 14 -ls lists files modified in the last 14 days
$ find . -name '*.bak' -ls -exec rm {} \; ls and then remove all files that end with .bak
![Page 24: Chapter 3 UNIX Utilities for Power Users](https://reader034.vdocument.in/reader034/viewer/2022051402/568158be550346895dc6090d/html5/thumbnails/24.jpg)
Find Examples
$ find ch6 -name j* ch6/junk ch6/junk/junk.csh ch6/junk.csh
![Page 25: Chapter 3 UNIX Utilities for Power Users](https://reader034.vdocument.in/reader034/viewer/2022051402/568158be550346895dc6090d/html5/thumbnails/25.jpg)
Scheduling Command (cron)
crontab cronTabName
crontab -ler [useName] Allows you to schedule a series of jobs
Executed on a periodic basis
Uses a file with the following line format:
minute hour day month weekday command(0-59) (0-23) (1-31) (1-12) (1-7; 1=Monday) (any command)
![Page 26: Chapter 3 UNIX Utilities for Power Users](https://reader034.vdocument.in/reader034/viewer/2022051402/568158be550346895dc6090d/html5/thumbnails/26.jpg)
Scheduling Example
$ cat crontab.cron0 8 * * 1 echo Happy Monday Morning* * * * * echo One Minute passed30 15 1 * 1 mail users % Jan meeting at 3pm
minute hour day month weekday command(0-59) (0-23) (1-31) (1-12) (1-7; 1=Monday) (any command)
![Page 27: Chapter 3 UNIX Utilities for Power Users](https://reader034.vdocument.in/reader034/viewer/2022051402/568158be550346895dc6090d/html5/thumbnails/27.jpg)
Scheduling Example$ crontab crontab.cron
$ crontab -l# DO NOT EDIT THIS FILE - edit the master and reinstall.# (crontab.cron installed on Sat Jun 26 23:33:35 2007)# (Cron version -- $Id: crontab.c,v 2.13 1994/01/17 03:20:37 vixie Exp $)0 8 * * 1 echo Happy Monday Morning* * * * * echo One Minute passed30 15 1 * 1 mail users % Jan meeting at 3pm
$ crontab -runregister crontab file
![Page 28: Chapter 3 UNIX Utilities for Power Users](https://reader034.vdocument.in/reader034/viewer/2022051402/568158be550346895dc6090d/html5/thumbnails/28.jpg)
Scheduling Command (at)
at command allows you to schedule one-time commands/scripts
at -csm time [date [, year]] [+increment] [-f script]
-c option : use C-shell
-s option : use Bourne shell
-m option : send mail
time is specified as HH or HHMM followed by an optional AM/PM
![Page 29: Chapter 3 UNIX Utilities for Power Users](https://reader034.vdocument.in/reader034/viewer/2022051402/568158be550346895dc6090d/html5/thumbnails/29.jpg)
Scheduling Command (at)
at -r [jobId]+
at -l [jobId]+ -r option : remove entry from at queue
-l option : list entries in at queue
![Page 30: Chapter 3 UNIX Utilities for Power Users](https://reader034.vdocument.in/reader034/viewer/2022051402/568158be550346895dc6090d/html5/thumbnails/30.jpg)
Scheduling Command (at)
at -csm time [date [, year]] [+increment] [script] time is specified as HH or HHMM followed by an
optional AM/PM
date is spelled out using first 3 letters of day and/or month
keyword now can be used in place of time sequence; today and tomorrow can be used in place of date
stated time may be augmented by an increment (number followed by minutes/hours/days/weeks/months/years)
![Page 31: Chapter 3 UNIX Utilities for Power Users](https://reader034.vdocument.in/reader034/viewer/2022051402/568158be550346895dc6090d/html5/thumbnails/31.jpg)
Scheduling Example
$ cat at.csh #!/bin/csh echo at done > /dev/tty1
$ at now + 2 minutes -f at.csh
$ at -l lists the job here
You may program the script to reschedule itself as follows:
#!/bin/csh date > /dev/tty1 at now + 2 minutes -f at.csh
![Page 32: Chapter 3 UNIX Utilities for Power Users](https://reader034.vdocument.in/reader034/viewer/2022051402/568158be550346895dc6090d/html5/thumbnails/32.jpg)
Pattern Scanning and Processing
awk: utility that scans one or more files and performs an action on all lines that match a particular condition
The conditions and actions are specified in an awk program.
awk reads a line breaks it into fields separated by tabs/spaces
or other separators specified by -F option
![Page 33: Chapter 3 UNIX Utilities for Power Users](https://reader034.vdocument.in/reader034/viewer/2022051402/568158be550346895dc6090d/html5/thumbnails/33.jpg)
awk Command
awk program has one or more commands:
[condition] [ \{ action \} ]
where condition is one of the following: special tokens BEGIN or END
an expression involving logical operators, relational operators, and/or regular expressions
![Page 34: Chapter 3 UNIX Utilities for Power Users](https://reader034.vdocument.in/reader034/viewer/2022051402/568158be550346895dc6090d/html5/thumbnails/34.jpg)
awk Command awk [condition] [ \{ action \} ]
action is one of the following kinds of C-like statements if-else; while; for; break; continue
assignment statement: var=expression
print; printf;
next (skip remaining patterns on current line)
exit (skips the rest of the current line)
list of statements
![Page 35: Chapter 3 UNIX Utilities for Power Users](https://reader034.vdocument.in/reader034/viewer/2022051402/568158be550346895dc6090d/html5/thumbnails/35.jpg)
awk Command
accessing individual fields: $1, ..., $n refer to fields 1 thru n
$0 refers to entire line
built-in variable NF means number of fields
% awk -F: '{ print NF, $1 }' /etc/passwd
prints the number of fields and the first field in the /etc/passwd file
-F: means to use : as the field separator
![Page 36: Chapter 3 UNIX Utilities for Power Users](https://reader034.vdocument.in/reader034/viewer/2022051402/568158be550346895dc6090d/html5/thumbnails/36.jpg)
awk Command
BEGIN condition triggered before first line read
END condition triggered after last line read
FILENAME: built-in variable for name of file being processed
We will use this data in following examples:% cat /etc/passwdnobody:*:-2:-2:Unprivileged User:/:/usr/bin/falseroot:*:0:0:System Administrator:/var/root:/bin/sh...lp:*:26:26:Printing Services:/var/spool/cups:/usr/bin/false
![Page 37: Chapter 3 UNIX Utilities for Power Users](https://reader034.vdocument.in/reader034/viewer/2022051402/568158be550346895dc6090d/html5/thumbnails/37.jpg)
awk Example
% cat p2.awk BEGIN { print "Start of file: "} { print $1 " " $6 " " $7 } END { print "End of file", FILENAME }
% awk -F: -f p2.awk /etc/passwdStart of file: nobody / /usr/bin/falseroot /var/root /bin/sh...lp /var/spool/cups /usr/bin/falseEnd of file /etc/passwd
![Page 38: Chapter 3 UNIX Utilities for Power Users](https://reader034.vdocument.in/reader034/viewer/2022051402/568158be550346895dc6090d/html5/thumbnails/38.jpg)
awk Operators
built-in variable NR contains current line #
remember, “-F:” uses colon as separator
% cat p3.awk NR > 1 && NR < 4 { print NR, $1, $6, NF }
% awk -F: -f p3.awk /etc/passwd 2 root /var/root /bin/sh 73 daemon /var/root /usr/bin/false 7
![Page 39: Chapter 3 UNIX Utilities for Power Users](https://reader034.vdocument.in/reader034/viewer/2022051402/568158be550346895dc6090d/html5/thumbnails/39.jpg)
awk Variables% cat p4.awk BEGIN {print "Scanning file"}{ printf "line %d: %s\n", NR, $0 lineCount++; wordCount += NF; }END { printf "lines = %d, words = %d\n", lineCount, wordCount }
% awk -f p4.awk /etc/passwdScanning fileline 1: nobody:*:-2:-2:Unprivileged User:/:/usr/bin/falseline 2: root:*:0:0:System Administrator:/var/root:/bin/sh...line 37: lp:*:26:26:Printing Services:/var/spool/cups:/usr/bin/falselines = 37, words = 141
![Page 40: Chapter 3 UNIX Utilities for Power Users](https://reader034.vdocument.in/reader034/viewer/2022051402/568158be550346895dc6090d/html5/thumbnails/40.jpg)
awk Control Structures% cat p5.awk
{ for (i = NF; i >= 1; i--) printf "%s ", $i; printf "\n";}
% awk -f p5.awk /etc/passwdUser:/:/usr/bin/false nobody:*:-2:-2:Unprivileged Administrator:/var/root:/bin/sh root:*:0:0:System ... Services:/var/spool/cups:/usr/bin/false lp:*:26:26:Printing
![Page 41: Chapter 3 UNIX Utilities for Power Users](https://reader034.vdocument.in/reader034/viewer/2022051402/568158be550346895dc6090d/html5/thumbnails/41.jpg)
awk Condition Ranges
Condition ranges: two expressions separated by comma
awk performs action on every line from the first line that matches first expression
until line that matches second condition
% awk -F: ' /nobody/,/root/ {print $0}' /etc/passwdnobody:*:-2:-2:Unprivileged User:/:/usr/bin/falseroot:*:0:0:System Administrator:/var/root:/bin/sh
![Page 42: Chapter 3 UNIX Utilities for Power Users](https://reader034.vdocument.in/reader034/viewer/2022051402/568158be550346895dc6090d/html5/thumbnails/42.jpg)
awk Built-in Functions
Built-in functions: exp()
log()
sqrt()
substr() etc.
% awk -F: '{print substr($1,1,2)}' /etc/passwdnoro...lp
![Page 43: Chapter 3 UNIX Utilities for Power Users](https://reader034.vdocument.in/reader034/viewer/2022051402/568158be550346895dc6090d/html5/thumbnails/43.jpg)
Hard Links
% ln original newLink Creates a hard link to original file called newLink
Both labels will refer to the same file
File will be deleted only when both labels are removed
If newLink is a directory then links are made within the directory
![Page 44: Chapter 3 UNIX Utilities for Power Users](https://reader034.vdocument.in/reader034/viewer/2022051402/568158be550346895dc6090d/html5/thumbnails/44.jpg)
Hard Link Example% cat > oneThis is an example file.
% ln one two% cat twoThis is an example file.
% vi two # changed “an” to “a great”% cat twoThis is a great example file.
% cat oneThis is a great example file.
% ls -l two-rw-r--r-- 2 mweeks mweeks 31 Sep 14 03:26 two
![Page 45: Chapter 3 UNIX Utilities for Power Users](https://reader034.vdocument.in/reader034/viewer/2022051402/568158be550346895dc6090d/html5/thumbnails/45.jpg)
Soft (Symbolic) Links % ln -s original newLink
Symbolic/soft links can be created from one file system into another file system
Hard links are restricted to one file system
Use ls -lL to view link details which file it refers to
ls -l displays the contents of the symbolic link% ls -llrwxrwxrwx 1 root root 23 2006-09-22 20:26 javac -> j2sdk1.4.2_05/bin/javac% ls -lL-rwxrwxr-x 1 10016 users 66156 2004-06-04 00:32 javac% ls -l j2sdk1.4.2_05/bin/javac-rwxrwxr-x 1 10016 users 66156 2004-06-04 00:32 j2sdk1.4.2_05/bin/javac
![Page 46: Chapter 3 UNIX Utilities for Power Users](https://reader034.vdocument.in/reader034/viewer/2022051402/568158be550346895dc6090d/html5/thumbnails/46.jpg)
In Depth
Unix files consist of two parts: the data part and the filename part.
The data part is associated with something called an 'inode'. The inode carries the map of where the data is, the file permissions, etc. for the data.
![Page 47: Chapter 3 UNIX Utilities for Power Users](https://reader034.vdocument.in/reader034/viewer/2022051402/568158be550346895dc6090d/html5/thumbnails/47.jpg)
In Depth The filename part carries a name and an associated
inode number.
More than one filename can reference the same inode number; these files are said to be 'hard linked' together.
![Page 48: Chapter 3 UNIX Utilities for Power Users](https://reader034.vdocument.in/reader034/viewer/2022051402/568158be550346895dc6090d/html5/thumbnails/48.jpg)
In Depth a 'soft link' or a 'symbolic link' is a special file
type whose data part carries a path to another file.
![Page 49: Chapter 3 UNIX Utilities for Power Users](https://reader034.vdocument.in/reader034/viewer/2022051402/568158be550346895dc6090d/html5/thumbnails/49.jpg)
Substituting User
% su [-] userName If userName is not specified, root is assumed
Need access privileges for this
Requires password
% sudo command User can execute command as superuser
Requires password
![Page 50: Chapter 3 UNIX Utilities for Power Users](https://reader034.vdocument.in/reader034/viewer/2022051402/568158be550346895dc6090d/html5/thumbnails/50.jpg)
Compress/Uncompress
compress/uncompress
% compress fileName (.Z)
% uncompress fileName
gzip
% gzip fileName (.gz)
% gunzip fileName
crypt
% crypt key < sample.txt > sample.crypt (to crypt)
% crypt key < sample.crypt > sample.txt (to uncrypt)
key could be any string
![Page 51: Chapter 3 UNIX Utilities for Power Users](https://reader034.vdocument.in/reader034/viewer/2022051402/568158be550346895dc6090d/html5/thumbnails/51.jpg)
Stream Editor (sed)
sed scans one or more text files
performs an edit on all lines that match a condition
actions and conditions may be stored in a file
may be specified at command line in single quotes
commands begin with an address or an addressRange or a Regular expression
does not modify the input file
writes modified file to standard output
![Page 52: Chapter 3 UNIX Utilities for Power Users](https://reader034.vdocument.in/reader034/viewer/2022051402/568158be550346895dc6090d/html5/thumbnails/52.jpg)
Things You Can Do with sed
a\ append lines to output until one not ending in \
b label branch to command : label
c\ change lines to following text, as in a\
d delete lines
i\ insert following text before next output
l list line, making all non-printing characters visible
(tabs appear as >; lines broken with \)
P print line
q quit (for scripts)
r file read file, copy contents to stdout
![Page 53: Chapter 3 UNIX Utilities for Power Users](https://reader034.vdocument.in/reader034/viewer/2022051402/568158be550346895dc6090d/html5/thumbnails/53.jpg)
Things You Can Do with sed s/pat1/pat2/f substitute pat2 for pat1
f = g, replace all occurrences f = p, print f = w file, write to file
t label test: branch to label if substitution made to current line
w file write line(s) to file
y/str1/str2/ replace each character from str1 with corresponding character from str2, no ranges allowed
= print current input line number
!cmd do sed cmd if line is not selected
: label set label for b and t commands
![Page 54: Chapter 3 UNIX Utilities for Power Users](https://reader034.vdocument.in/reader034/viewer/2022051402/568158be550346895dc6090d/html5/thumbnails/54.jpg)
Ranges by line number
You can specify a range on line numbers by inserting a comma between the numbers. To restrict a substitution to the first 100 lines, you can use: sed '1,100 s/A/a/'
If you know exactly how many lines are in a file, you can explicitly state that number to perform the substitution on the rest of the file. In this case, assume you used wc to find out there are 532 lines in the file: sed '101,532 s/A/a/'
An easier way is to use the special character "$," which means the last line in the file.
sed '101,$ s/A/a/'
The "$" is one of those conventions that mean "last" in utilities like cat -e, vi, and ed. "cat -e" Line numbers are cumulative if several files are edited. That is, sed '200,300 s/A/a/' f1 f2 f3 >newis the same as cat f1 f2 f3 | sed '200,300 s/A/a/' >new
![Page 55: Chapter 3 UNIX Utilities for Power Users](https://reader034.vdocument.in/reader034/viewer/2022051402/568158be550346895dc6090d/html5/thumbnails/55.jpg)
Ranges by patterns
You can specify two regular expressions as the range. Assuming a "#" starts a comment, you can search for a keyword, remove all comments until you see the second keyword. In this case the two keywords are "start" and "stop:" sed '/start/,/stop/ s/#.*//' The first pattern turns on a flag that tells sed to perform the substitute command on every line. The second pattern turns off the flag. If the "start" and "stop" pattern occurs twice, the substitution is done both times. If the "stop" pattern is missing, the flag is never turned off, and the substitution will be performed on every line until the end of the file.
![Page 56: Chapter 3 UNIX Utilities for Power Users](https://reader034.vdocument.in/reader034/viewer/2022051402/568158be550346895dc6090d/html5/thumbnails/56.jpg)
Substituting Text
% sed 's/^/ /' file > file.new
indents each line in the file by 2 spaces
% sed 's/^ *//' file > file.new
removes all leading spaces from each line of the file
% sed '/a/d' file > file.new
deletes all lines containing 'a'
Example: add two lines in the beginning of the file
% cat sed1 1i\ abcd\ efg % sed -f sed1 file > file.new
![Page 57: Chapter 3 UNIX Utilities for Power Users](https://reader034.vdocument.in/reader034/viewer/2022051402/568158be550346895dc6090d/html5/thumbnails/57.jpg)
Example Using sed
Replace lines 1-3 by “Lines 1-3 are censored”% cat sed21,3c\Lines 1-3 are censored
% cat dummy2onetwo three four five six
% sed -f sed2 dummy2Lines 1-3 are censoredfour five six
![Page 58: Chapter 3 UNIX Utilities for Power Users](https://reader034.vdocument.in/reader034/viewer/2022051402/568158be550346895dc6090d/html5/thumbnails/58.jpg)
Example Using sed
Multiple commands; replaces individual lines
% cat sed31c\Line 1 is censored2c\Line 2 is censored3c\Line 3 is censored
% sed -f sed3 dummy2Line 1 is censoredLine 2 is censoredLine 3 is censoredfour five six
![Page 59: Chapter 3 UNIX Utilities for Power Users](https://reader034.vdocument.in/reader034/viewer/2022051402/568158be550346895dc6090d/html5/thumbnails/59.jpg)
Example Using sed
% sed '$r file' f > g appends file at end of file f
% sed -e 's/^/<< /' -e 's/$/ >>/' file multiple commands
-e option is optional! means script on command line
% sed -e 's/^/<< /' -e 's/$/ >>/' dummy2<< one >><< two >><< three >><< four >><< five >><< six >>
![Page 60: Chapter 3 UNIX Utilities for Power Users](https://reader034.vdocument.in/reader034/viewer/2022051402/568158be550346895dc6090d/html5/thumbnails/60.jpg)
Translate Utility (tr)
% tr -cds string1 string2
maps all characters in std. input from character set string1 to the corresponding character in string2
If length of string2 is less, the last character is repeated.
![Page 61: Chapter 3 UNIX Utilities for Power Users](https://reader034.vdocument.in/reader034/viewer/2022051402/568158be550346895dc6090d/html5/thumbnails/61.jpg)
Translate Utility (tr)
% tr -cds string1 string2 -c option causes string1 to be complemented (every
character not in string1 now is in string1!)
-d option causes every character in string1 to be deleted from std. input
-s option causes every repeated character in string1 to be condensed to one occurrence
![Page 62: Chapter 3 UNIX Utilities for Power Users](https://reader034.vdocument.in/reader034/viewer/2022051402/568158be550346895dc6090d/html5/thumbnails/62.jpg)
Examples of tr % tr a-z A-Z < file1 > file2
causes lower-case to upper-case conversion % tr -c a X < file1 > file2
causes every non-a character to be replaced by X (including newline)
% tr -c a-z '\012' < file1 > file2
causes all non alphabetic characters to be replaced by ASCII 12 (newline)
% tr -d a-c < file1 > file2
causes all a, b, c to be deleted from file1
![Page 63: Chapter 3 UNIX Utilities for Power Users](https://reader034.vdocument.in/reader034/viewer/2022051402/568158be550346895dc6090d/html5/thumbnails/63.jpg)
Perl(Practical Extraction and Report Language)
Perl takes its syntax and features from Shell scripts
C programming
Printing text: print "hello world\n";
Variables: (untyped) always begins with a $ $i = 3;
http://learn.perl.org/books.html
![Page 64: Chapter 3 UNIX Utilities for Power Users](https://reader034.vdocument.in/reader034/viewer/2022051402/568158be550346895dc6090d/html5/thumbnails/64.jpg)
Perl Range Operators
In addition to standard arithmetic operators, a range operator is available.
print 1, 2, 3..15, "\n"; # numbers, range
print "AB", "BBBC", "CDD", "\n"; # strings
print "A"."B"; # concatenation
![Page 65: Chapter 3 UNIX Utilities for Power Users](https://reader034.vdocument.in/reader034/viewer/2022051402/568158be550346895dc6090d/html5/thumbnails/65.jpg)
Perl Arrays
Arrays: dynamically allocated
Array names begin with @ symbol
Array index begins at 0
@arr = (1,2,3,4,5);
@brr = (1..15);
print @arr[3], "\n";
print @arr, "\n"; # will print 12345
print @arr + @brr; # will print 20
![Page 66: Chapter 3 UNIX Utilities for Power Users](https://reader034.vdocument.in/reader034/viewer/2022051402/568158be550346895dc6090d/html5/thumbnails/66.jpg)
Perl Math and Logic
Mathematical and Logical operators +, -, *, / ++, -- &&, ||
![Page 67: Chapter 3 UNIX Utilities for Power Users](https://reader034.vdocument.in/reader034/viewer/2022051402/568158be550346895dc6090d/html5/thumbnails/67.jpg)
Perl Loops and Conditional Processing
if-else, while, for (from C) and foreach (from C shell)
if ($i == 0) { print "it's true\n";}else { print "it's false\n";}
while ($i <= $j) { print "it's true\n"; $i++;}
![Page 68: Chapter 3 UNIX Utilities for Power Users](https://reader034.vdocument.in/reader034/viewer/2022051402/568158be550346895dc6090d/html5/thumbnails/68.jpg)
Perl Loops and Conditional Processing
if-else, while, for (from C) and foreach (from C shell)
for ($i=0; $i<10; $i++) { print "\$i=",$i, "\n";}
foreach $i (1..15) { print "\$i=",$i, "\n";}
![Page 69: Chapter 3 UNIX Utilities for Power Users](https://reader034.vdocument.in/reader034/viewer/2022051402/568158be550346895dc6090d/html5/thumbnails/69.jpg)
Perl File I/O @line = <stdin>; foreach $i (@line) { print "->", $i; # also reads in EOLN; # reads and prints each line # from stdin } $FILE = "info.dat"; open(FILE); @arr = <FILE>; # an array of lines close(FILE); foreach $line (@arr) { print "$line"; }
![Page 70: Chapter 3 UNIX Utilities for Power Users](https://reader034.vdocument.in/reader034/viewer/2022051402/568158be550346895dc6090d/html5/thumbnails/70.jpg)
Perl Functionssub pound2dollars { $EXCHANGE_RATE = 1.54; $pounds = $_[0]; # refers to 1st argument return ($EXCHANGE_RATE * $pounds);}
$book = 3.0;$value = pound2dollars($book);print "Value in dollars = $value\n";
![Page 71: Chapter 3 UNIX Utilities for Power Users](https://reader034.vdocument.in/reader034/viewer/2022051402/568158be550346895dc6090d/html5/thumbnails/71.jpg)
Command Line Arguments to Perl$n = $#ARGV+1; # number of args (beginning 0)print $n, " args: \n";for ($i = 0; $i < n; $i++) { print "@ARGV[$i]\n";}
# pounds on command line:
if ($#ARGV < 0) { print "Specify value in pounds to convert to dollars\n"; exit;}$poundvalue = @ARGV[0];$dollarvalue = pounds2dollars($poundvalue);print "Value in dollars = $dollarvalue\n";
![Page 72: Chapter 3 UNIX Utilities for Power Users](https://reader034.vdocument.in/reader034/viewer/2022051402/568158be550346895dc6090d/html5/thumbnails/72.jpg)
Perl Example
loan -a amount -p payment -r rate
prints a table showing each month; principal and interest and balance
$i = 0;while ($i < $#ARGV) { if (@ARGV[$i] eq "-r") { $RATE = @ARGV[++$i]; } else { if (@ARGV[$i] eq "-a") { $AMOUNT = @ARGV[++$i]; } else { if (@ARGV[$i] eq "-p") { $PAYMENT = @ARGV[++$i]; } else { print "Unknown argument (@ARGV[$i])\n"; exit } } } $i++;}
![Page 73: Chapter 3 UNIX Utilities for Power Users](https://reader034.vdocument.in/reader034/viewer/2022051402/568158be550346895dc6090d/html5/thumbnails/73.jpg)
Perl Example (continued)
if ($AMOUNT == 0 || $RATE == 0 || $PAYMENT == 0) { print "Specify -r rate -a amount -p payment\n"; exit}
print "Original balance: \$$AMOUNT\n";print "Interest rate : ${RATE}%\n";print "Monthly payment : \$$PAYMENT\n";print "\n";print "Month\tPayment\tInterest\tPrincipal\tBalance\n\n"; $month = 1;$rate = $RATE/12/100;$balance = $AMOUNT;$payment = $PAYMENT;
![Page 74: Chapter 3 UNIX Utilities for Power Users](https://reader034.vdocument.in/reader034/viewer/2022051402/568158be550346895dc6090d/html5/thumbnails/74.jpg)
Perl Example (continued)while ($balance > 0) { $interest = roundUpAmount($rate * $balance); $principal = roundUpAmount($payment - $interest); if ($balance < $principal) { $principal = $balance; $payment = $principal + $interest; } $balance = roundUpAmount($balance - $principal); print "$month\t\$$payment\t\$$interest\t\t\$$principal\t\t\$$balance\n"; $month++;}
sub roundUpAmount { $value = $_[0]; $newvalue = (int (($value * 100) + 0.5))/100; return $newvalue;}
![Page 75: Chapter 3 UNIX Utilities for Power Users](https://reader034.vdocument.in/reader034/viewer/2022051402/568158be550346895dc6090d/html5/thumbnails/75.jpg)
Perl Example (continued)$ perl loan.pl -r 12.5 -p 30 -a 300Original balance: $300Interest rate : 12.5%Monthly payment : $30
Month Payment Interest Principal Balance
1 $30 $3.13 $26.87 $273.132 $30 $2.85 $27.15 $245.983 $30 $2.56 $27.44 $218.544 $30 $2.28 $27.72 $190.825 $30 $1.99 $28.01 $162.816 $30 $1.7 $28.3 $134.517 $30 $1.4 $28.6 $105.918 $30 $1.1 $28.9 $77.019 $30 $0.8 $29.2 $47.8110 $30 $0.5 $29.5 $18.31
![Page 76: Chapter 3 UNIX Utilities for Power Users](https://reader034.vdocument.in/reader034/viewer/2022051402/568158be550346895dc6090d/html5/thumbnails/76.jpg)
Review
Advanced commands
Pattern matching (grep)
Sort
Comparing (cmp, diff)
Archiving (tar)
Scheduling (cron, at)
Pattern matching and processing (awk, sed)
Perl