chapter 4: unix file processing input and output
TRANSCRIPT
![Page 1: Chapter 4: UNIX File Processing Input and Output](https://reader036.vdocument.in/reader036/viewer/2022081421/56649de45503460f94adad92/html5/thumbnails/1.jpg)
Chapter 4: UNIX File Processing
Input and Output
![Page 2: Chapter 4: UNIX File Processing Input and Output](https://reader036.vdocument.in/reader036/viewer/2022081421/56649de45503460f94adad92/html5/thumbnails/2.jpg)
Extracting Information from FilesSection A: Objectives
After studying this lesson, you should be able to understand and use:– Redirection– Pipes– Wildcards and Regular Expressions– Commands:
find, sort, catgrep
![Page 3: Chapter 4: UNIX File Processing Input and Output](https://reader036.vdocument.in/reader036/viewer/2022081421/56649de45503460f94adad92/html5/thumbnails/3.jpg)
Processing Files
• When performing commands, UNIX processes data, it receives input from the standard input device (the keyboard) and then sends output to the standard output device (the screen).
• System administrators and programmers refer to standard input as stdin
• They refer to standard output as stdout• The third standard output is called standard
error, or stderr (also typically the screen), where error messages are displayed
![Page 4: Chapter 4: UNIX File Processing Input and Output](https://reader036.vdocument.in/reader036/viewer/2022081421/56649de45503460f94adad92/html5/thumbnails/4.jpg)
Using Input and Error Redirection
• File Redirection operators
> - redirect output, send it to a file
>> - redirect output, appends it to a file
< - redirects input, retrieves input from a file
![Page 5: Chapter 4: UNIX File Processing Input and Output](https://reader036.vdocument.in/reader036/viewer/2022081421/56649de45503460f94adad92/html5/thumbnails/5.jpg)
Redirecting Outputs
• $ ls > foo• Send the directory list to the file named
“foo”
• $ cat foo• Show contents of “foo”
• $ date >> foo• Append the date to the file name “foo”
![Page 6: Chapter 4: UNIX File Processing Input and Output](https://reader036.vdocument.in/reader036/viewer/2022081421/56649de45503460f94adad92/html5/thumbnails/6.jpg)
Redirecting Outputs (cont)
$ ls -l > barSend the long directory list to “bar”
$ cat foo bar > foobarCombine “foo” and “bar” into a new file
name “foobar”
$ cat foobarDisplay contents of new file
![Page 7: Chapter 4: UNIX File Processing Input and Output](https://reader036.vdocument.in/reader036/viewer/2022081421/56649de45503460f94adad92/html5/thumbnails/7.jpg)
Redirecting Inputs
$ who > peoplePut the list of users currently on the system
into the file “people”
$ sort < peopleSort by sending “people” as input to sort
$ sort < people > speopleSend sorted list to new file “speople”
![Page 8: Chapter 4: UNIX File Processing Input and Output](https://reader036.vdocument.in/reader036/viewer/2022081421/56649de45503460f94adad92/html5/thumbnails/8.jpg)
Using Pipes
• The pipe operator (|) redirects the output of one command to the input of another command
• The pipe operator is used in the following manner:first_command | second_command
• The pipe operator connects the output of the first command with the input of the second command
• The pipe operator can connect several commands on the same command line, in the following manner:first_command | second_command | third_command ...
![Page 9: Chapter 4: UNIX File Processing Input and Output](https://reader036.vdocument.in/reader036/viewer/2022081421/56649de45503460f94adad92/html5/thumbnails/9.jpg)
Pipes Examples
$ who > people
$ sort < people
Is there a shortcut? Yes, using pipes.
$ who | sort List of users on the system
$ who | wc –l How many users are on the system?
![Page 10: Chapter 4: UNIX File Processing Input and Output](https://reader036.vdocument.in/reader036/viewer/2022081421/56649de45503460f94adad92/html5/thumbnails/10.jpg)
Wildcards
The ‘*’ and ‘?’ characters are wildcards, which match against zero or more characters in a file or directory name.
‘*’ matches any # of characters (0 or more).
‘?’ matches exactly one character
[ ] and - : character range, a way to specify a sub range of characters to match.
![Page 11: Chapter 4: UNIX File Processing Input and Output](https://reader036.vdocument.in/reader036/viewer/2022081421/56649de45503460f94adad92/html5/thumbnails/11.jpg)
Wildcard Examples
$ ls foo* Lists all files starting with “foo”$ ls *.c Lists all files ending with “.c”$ ls f.? Lists all files named “f” plus an extra
character, such as “f.1” or “f.c”$ ls a* d* f* List all files starting with a, d, f$ ls [adf]* same$ ls [a-z]* List files starting with any letter$ ls [a-z] * List all one letter files, then all
files
![Page 12: Chapter 4: UNIX File Processing Input and Output](https://reader036.vdocument.in/reader036/viewer/2022081421/56649de45503460f94adad92/html5/thumbnails/12.jpg)
Wildcard Examples (cont)
$ ls *unix* list all files/subdirectories which contain
the word “unix”
$ ls *s*the following example files will match this
pattern.grades.txt, prog1.s, summary.doc
$ ls *[0-9._]* files having 0-9 or . or _
![Page 13: Chapter 4: UNIX File Processing Input and Output](https://reader036.vdocument.in/reader036/viewer/2022081421/56649de45503460f94adad92/html5/thumbnails/13.jpg)
Wildcard Examples (cont)
$ rm *Delete ALL files
$ rm ???Delete all filenames with 3 characters
$ cp *.txt ..Copy all files with a “.txt” extension to the
parent directory
![Page 14: Chapter 4: UNIX File Processing Input and Output](https://reader036.vdocument.in/reader036/viewer/2022081421/56649de45503460f94adad92/html5/thumbnails/14.jpg)
Wildcard Examples (cont)
• Example: to list any file that start with a lowercase letter range from a to z as specified
ls -l [a-z]*
• Which of the following files will this pattern match.ls hw[0-9]*For example: hw1, hw3.c, hw50.txt, hw.
![Page 15: Chapter 4: UNIX File Processing Input and Output](https://reader036.vdocument.in/reader036/viewer/2022081421/56649de45503460f94adad92/html5/thumbnails/15.jpg)
Finding Files
• The find command searches for files that have a specified name
• The command has the form:
find pathname - name filename• The find command prohibits you from searching
where you do not have system-level permissions • Although Linux does not use it, other UNIX
versions require the -print option after the filename to display the names of fields the find command locates
![Page 16: Chapter 4: UNIX File Processing Input and Output](https://reader036.vdocument.in/reader036/viewer/2022081421/56649de45503460f94adad92/html5/thumbnails/16.jpg)
Finding Files Examples
• find /usr -name Chapter1 -type f -print search through the "/usr" directory for the file
named "Chapter1".
• find /usr -name "Chapter*" -type f -print search through the "/usr" directory for all files that
begin with the letters "Chapter". The filename can end with any other combination of characters.
This will match filenames such as "Chapter", "Chapter1", "Chapter1.bad", "Chapter_in_life".
![Page 17: Chapter 4: UNIX File Processing Input and Output](https://reader036.vdocument.in/reader036/viewer/2022081421/56649de45503460f94adad92/html5/thumbnails/17.jpg)
Using the Sort Command
• Use the sort command to sort a file’s contents alphabetically or numerically
• Here is an example of its use:
sort file1 > file2
• Here are more complex examples:
ls –s | sort –n list files in sorted order
ls –s | sort –nr reverse sorted order
ls –s | sort –nr | head -5 show 5 largest files
![Page 18: Chapter 4: UNIX File Processing Input and Output](https://reader036.vdocument.in/reader036/viewer/2022081421/56649de45503460f94adad92/html5/thumbnails/18.jpg)
grep
• grep command is used to match a pattern in one or more files, and displays the matching output on standard output.
• The format of grep command is:grep [options] regexp [files]
• grep means global regular expression print.
![Page 19: Chapter 4: UNIX File Processing Input and Output](https://reader036.vdocument.in/reader036/viewer/2022081421/56649de45503460f94adad92/html5/thumbnails/19.jpg)
grep
• Example: grep luo /etc/passwd: to search a user in the system who called 'luo' as the first name or the last name.
• Quotes (single or double) are needed if regular expressions are to be usedgrep olga* will stallgrep ‘olga*’ will work fine
• Wildcards in filename can be used with grep as usualgrep ‘olga*’ *.txt
![Page 20: Chapter 4: UNIX File Processing Input and Output](https://reader036.vdocument.in/reader036/viewer/2022081421/56649de45503460f94adad92/html5/thumbnails/20.jpg)
Regular Expressions
• b* ( , b, bb, bbb, bbbb, ….) • . matches any single character • ^ beginning of the line
grep ‘^taylor’ file1 find pattern ^taylor• $ end of the line
grep ‘:$’ file1 entries that ends with :
grep ‘100\$’ f1 searches for 100$
grep ‘100$’ f1 searches for 100 at the end
![Page 21: Chapter 4: UNIX File Processing Input and Output](https://reader036.vdocument.in/reader036/viewer/2022081421/56649de45503460f94adad92/html5/thumbnails/21.jpg)
Regular Expressions
• [xy] any single character in the set• [^xy] any single character not in the set
grep ‘^taylor[a-z]’ file1 starts with taylor and contains a lower case letter afterwardsgrep ‘^taylor[^a-z]’ file1 starts with taylor and no lower case letter afterwardsgrep ‘z*’ f1 zero or more occurrences of z (everything in f1)grep ‘zz*’ f1 one or more occurrences of z (does not return lines without z)
![Page 22: Chapter 4: UNIX File Processing Input and Output](https://reader036.vdocument.in/reader036/viewer/2022081421/56649de45503460f94adad92/html5/thumbnails/22.jpg)
Grep flags
• -c: print a count of matched lines only• -l: list filenames that contain the specified
pattern• -i: Ignore the case of the letters in the pattern• -n: Including line numbers• -b: precede each line with its block number• -h: print matched lines but not
filenames(inverse of -l)• -s: Suppress error messages for nonexistent of
unreadable• -v: Print all lines that don't match regexp
![Page 23: Chapter 4: UNIX File Processing Input and Output](https://reader036.vdocument.in/reader036/viewer/2022081421/56649de45503460f94adad92/html5/thumbnails/23.jpg)
grep
• Example 1: to indicate how many users on the system has a last name or first name as "marylin" ignoring the upper or lower cases.grep -ci marylin /etc/passwd
• Example2: to indicate how may case-sensitive matches are in the file before any of them are displayed on the screengrep -c lu /etc/passwd
• Example3: to number the output lines.grep -n lu /etc/passwd
![Page 24: Chapter 4: UNIX File Processing Input and Output](https://reader036.vdocument.in/reader036/viewer/2022081421/56649de45503460f94adad92/html5/thumbnails/24.jpg)
Chapter Summary
• Output from a command may be redirected from stdout to a disk file
• Input to a command may be redirected from stdin to a disk file
• Use the sort command to sort a file’s contents alphabetically or numerically
• A pipe directs the standard output of one Unix command into the standard input of another Unix command.
• A wildcard is a tool that allow you to specify a group of filenames or directories.
• Useful commands such as find, grep, cat,