10/18/2015course material created by d. woit 1 cps 393 introduction to unix and c start of week 2...

04/20/23 Course material created by D. Woit

1

CPS 393Introduction to Unix and C

START OF WEEK 2 (UNIX)

04/20/23 Course material created by D. Woit 1


2

Glob Constructs (Metacharacters) and other Special Characters

• ? * [ ] #constructs– saves typing, time– encourages good naming conventions

– when shell sees a glob construct, it first *expands* word containing

construct, then does command.• ? any single character• * any string of characters (incl. null string)• [...] any single char of series of chars within the brackets• e.g., assume /home/dwoit has 7 files: • lab1.jav lb2.jav new.c pie.c prog1.c prog2.c top.c • /home/dwoit> ls *.c• new.c pie.c prog1.c prog2.c top.c



3

(Metacharacters) and other Special Characters cont.

• /home/dwoit> ls *

lab1.jav lb2.jav new.c pie.c prog1.c prog2.c top.c

• /home/dwoit> ls prog?.c

prog1.c prog2.c• /home/dwoit> ls p*.c

prog1.c prog2.c pie.c• /home/dwoit> ls ???.*

lb2.jav new.c pie.c top.c• /home/dwoit> ls prog[123].c

prog1.c prog2.c04/20/23 Course material created by D. Woit 3


4

Metacharacters and other Special Chars cont.

• /home/dwoit> ls [m-s]* # all files starting with m ... s new.c pie.c prog1.c prog2.c

• [1-5A-Z] matches any char 1,2,3,4,5,A,B,C, ... Z• [!ABC] matches any char *except* A, B or C• [!a-z0-47] matches any char *except* a,b,c, ... z, 0,1,2,3,4,7

– NOTE: ranges, such as a-z depend on locale having LC_COLLATE set normally.

– It can be set to something weird, such as thinking order is aAbBcC...zZ instead of the normal ASCII sequence abc..zABC...Z

– You can do a simple test to check it out: e.g., if ls [a-z] matches A it is not normal. (in latest bash, either ! or ^ is not)



5

Other special characters, expanded to • ~ your home directory name (e.g. /home/dwoit )

– from any dir, ls ~ lists your home dir (/home/jmisic for me)

– cd ~/cps393 changes into dir /home/jmisic/cps393 for me • ~usr the home dir of user with userid "usr "• ~- your previous working dir (note in bash cd - == cd ~-)• ~+ your current working dir

• Example: toggle between 2 dirs with "cd -" in bash

– /home/dwoit> cd bin– /home/dwoit/bin> cd -– /home/dwoit> cd -– /home/dwoit/bin>



6

Directories and files

• /home/jmisic/cps393> echo my working dir is ~+– my working dir is /home/jmisic/cps393

• /home/jmisic/cps393/labs> echo all the files in this dir are *– all the files in this dir are lab1.txt

• However, if we put the quotes we get:• /home/jmisic/cps393/labs> echo “all the files in this dir are *”

– echo all the files in this dir are *

• /home/jmisic/cps393/labs> echo all the files in this dir are “*”– echo all the files in this dir are *



7

Directories and files cont.• Special chars are not expanded if within single or double

quotes

• escape character \ before any special char works like quotes

• /home/jmisic/cps393/labs> echo all the files in this dir are \*– echo all the files in this dir are *



8

Patterns and Regular Expressions in Shell • in ksh they work normally• -in bash if you execute first: shopt -s extglob (extended

pattern matching)– Without this option only *, ? And […] work.

• For example:• *(exp) 0 or more occurrences of exp• +(exp) 1 or more occurrences of exp• ?(exp) 0 or 1 occurrences of exp• @(exp1|exp2|...) exp1 or exp2 or ...• !(exp) anything that doesn't match exp



9

Patterns and Regular Expressions in Shell

• Example: if you had files in the current dir such as below:– A Ax Axxx Axxxx X X.bak x xx xxxx

• Then (but carefully since no extra spaces are allowed in braces):

• shopt -s extglob• ls A*(x) prints A Ax Axxx Axxxx• ls A*(xx) prints A Axxxx• ls A+(x) prints Ax Axxx Axxxx• ls A?(x) prints A Ax• ls X?(.bak) prints X X.bak• ls @(*xx|*ak) prints Axxx Axxxx X.bak xx xxxx

• ls !(@(*xx|*ak)) prints A Ax X x



10

Homework• How would you list all file/dir names that: • 1. contained the letter "e"; • 2. had a total of 7 letters and no extension; • 2a. are 7 characters long;• 3. had 3 letter extensions; • 3a. had 3 letter alpha-numeric extensions;

• 4. contained the word "tst" *somewhere* in the name;

• 5. started with the letter "A" and contained a dash (-);• 6. are at least 2 characters long and don't have the following letters

anywhere in the first 2 characters: a,b,c,f,h,x,y,z• 7. end in .ex with an optional c at the end• 8. are not java or text files (don't end in .jav or .txt)• 9. end in a : followed by a digit from 2-8 followed zero or more digits.



11

Interactive Shell Use

• Is usually used for– typing on command line– one-time task– Prototyping

• For repetitive use we build Shell scripts (Shell programs)– task complex– repeat it later



12

Example shell script• #!/bin/bash• #source: showSource• #shell program to list all source java and c files in current directory

• echo -n "Current directory is: "• pwd• echo "C files are:"• ls *.c• echo "Java files are:"• ls *.jav• To run: make sure it is executable (chmod ugo+x

showSource or chmod +x showSource ), then• jmisic@metis:~/cps393/labs$ ./showSource



13

Example output of the script is:

• jmisic@metis:~/cps393/labs$ ./showSource• current directory is/home/jmisic/cps393/labs• c files are:• smitz5.c smitz_scanf.c• java files are:• proba.jav


14

Arguments to shell script: $1 $2 $3 etc.

• $0 name of shell pgm• $# count of # of args passed• $@ lists args as one string: "arg1 arg2 ...“• $* lists args as separate : "arg1" "arg2" ...• Example:• #!/bin/bash• #source: prnargs• #shell pgm to print out its first 2 args• echo The first argument is: $1• echo The second argument is: $2• exit 0


15

Arguments: $1 $2 $3 etc.

• To run set execute permissions:• jmisic@metis:~/cps393/labs$ ./prnargs "Hello there " world

– The first argument is: Hello there– The second argument is: world

• Note:• could have used quotes in echo:

– echo "The first argument is: " $1– echo "The second argument is:" $2

• or could have used echo backslash-escape TAB char, \t (see man echo)– echo -e "The first argument is:\t" $1– echo -e "The second argument is:\t" $2


16

How to leave out the ./• current dir has to be in your path• If you wish to invoke prnargs (instead of ./prnargs )• you need to edit your .profile file in your home directory

(but save it first)• Since current directory is represented as null name

followed by a colon you have to insert at the end :– PATH=$PATH::– export PATH

• Export command makes PATH variable available to all subsequent processes initiated by the user.


17

Filters• Filter refines or transforms the input to produce (usually

different) output

• stdin -------> filter --------> stdout

• E.g. more < file1

– transforms sequence of characters from file1 (its input)

– into the same sequence but with pagination


18

Grep (globally look for a regular expression and print)

• grep string filename(s)– string: the string to search for– filename(s): one or more files to search in– result: displays lines of file(s) filename(s) that contain the given string– (precisely: sends to stdout lines of stdin that contain string)

• e.g. grep 'applet' lab1.jav lab2.jav• sends to stdout (screen) those lines of files lab1.jav and• lab2.jav that contain the string applet• /home/jchan> grep 'applet' lab1.jav lab2.jav• lab1.jav: applet.init();• lab1.jav: applet.start();• lab2.jav: applet.num=6;


19

Grep cont.• e.g. You forgot the exact syntax for 'for' construct in java. • You know it is used in one of your .jav files. Do:• grep 'for' *.jav

– to get a list of all files/lines containing for – Then edit or more one file to see exact syntax

• Note: – single quotes are good for now. Once we study variables, you– we will have reason to use double quotes around the search

string.

• Options for grep• -i option: ignore case of search string• -v option: print lines *not* matching search string


20

Grep with regular expressions and metacharacters

• . any character, except newline (like ? in glob)• * 0 or more repetitions of previous character (not like glob!)• ^ beginning of line• $ end of line• [ ... ] any character inside the brackets (like glob)• [^ ... ] any character *not* inside the brackets (like ! or ^ in glob)• [! ... ] same as above on some implementations• \{m\} exactly m repetitions of previous character• \{m,n\} any number of repetitions of prev char between m and n

inclusive • \< beginning of word• \> end of word• remember to quote these as in: grep '\<a_word\>' a_file


21

Examples: Grep with special chars.• grep '^Assignment' fname # lines starting with Assignment• grep -v '^Assignment' fname # lines not starting with

Assignment• grep 'Assignment$' fname # lines ending with Assignment• grep 'd.g' fname # lines containing dag, dbg, dcg, d0g,

d1g, etc• grep 'su*m' fname # lines containing sm, sum, suum,

suuum, etc.• grep '\<so\>' fname # *word* so (vs. social, absolute)• grep '[A-Za-z][A-Za-z]*' fname # lines containing ANY non-

empty alpha string (no blank lines, lines with digits, etc)


22

Examples: Grep with special chars• grep 'xyz\.[^ ]* ' # lines containing string xyz followed

by a dot followed by 0 or more non-spaces, then one space e.g., "xyz. " "xyz.w " "xyz.ta " etc (see slide 20)

• grep 'xyz\.w\{2,3\}X' # lines containing xyz.wwX or xyz.wwwX


23

Grep with Extended Regular Expressions:

• use grep -E or egrep (system dependent) OR • + 1 or more repetitions of previous character • e.g., • grep -E 'dog|cat|bird' fname # lines with at least 1 of dog,

cat, or bird• grep -E '[A-Za-z]+' fname # lines with at least one alpha

char• Note this also matches lines with no alpha chars since *

matches null:

• grep '[A-Za-z]*' fname


24

Homework• In the shell, is grep '[A-Za-z][A-Za-z]*' fname the same asgrep -i '[A-Z]*' fname ? Why or why not?• Why do you think "\<" and \>" are used instead of simply "<" and ">"

for beginning/end of line? • For questions 1-6 below, write a different shell program. 1. list all lines in file fname containing the string "dog"; 2. list all lines in fname containing the word "dog"; 3. list all lines in fname containing the string "dog" at the beginning of a

line; 4. list all lines in fname containing the word "dog" at the beginning of a

line.5. list all lines in fname containing exactly 6 "a"s in a row.6. list all lines in fname containing one or more words that start with 93

and end in a sequence of any number of W (not 0)7. Find 3 different grep commands that will list lines of fname that end

in a vowel. 8. Use the man page to find the option of grep that shows the matched

string in a different color. Try it.


25

find (list all files and dirs meeting certain criteria )

• e.g., this lists all java files in portion of the file system rooted at /home/dwoit:

• find /home/dwoit -name "*.jav" – match any file with name *.jav (glob)– dir to search (this dir and all its subdirectories )

• find . -type f # lists all *files* in filesystem

rooted at the current dir (.)

(i.e., no directories listed;

for dirs -type d)


26

find cont.

• find /usr -name banner 2>/dev/null

• find . -type d -perm -g+r,u+r # all dirs rooted in current dir that are readable by both owner and group LOTS of ways to specify perms--see man find


27

Notes for find• If you want it to go down symbolic links, you might have

to use the -follow option. • Some versions of linux have default find commands that

follow automatically, and others don't...• Some years scs students' home dirs are symbolic links.• Note: if glob constructs not in quotes, shell expands *first*

and then sends all to find. Again, single or double quotes OK for now.


28

HMWK1. Use the "find" command to list all files in directory /bin that start with

the letter "m". 2. What would happen if we did not put a name such as "*.jav" in quotes

in the find command? Make sure you have some .jav files in your directory and try it out.

3. Look at the man pages for "find" to determine what the option "-mtime" does in the find command. Then..Figure out a sequence of commands (involving a find, temp file and a grep) to display those files in the current directory whose contents werechanged within the last 24 hours and whose names contain the string "tst".

4. Make a shell program to do question 3 above. Call your program tstRecent. Your program should delete any temporary files it creates.Test it to make sure your program works.

5. Use the man pages to find out how find's -maxdepth option works. Use -maxdepth 1 to list all files in the given directory only (not the whole filetree)


29

Head, Tail

• head sends to stdout the first 10 lines of stdin • tail sends to stdout the last 10 lines of stdin • e.g., • head <myfile # displays first 10 lines of file• myfile on screen• tail <myfile >end.of.myfile # puts copy of last 10 lines of • myfile into file called end.of.myfile• option -number displays first/last "number" lines• e.g., head -20 f1 # first 20 lines of file f1• (options are in man pages. Lots for tail. )

10/18/2015course material created by d. woit 1 cps 393 introduction to unix and c start of week 2...

Documents