1

Day 16

Sed and Awk

2

Looking through output

• We already know what “grep” does.– It looks for something in a file.– Returns any line from the file that contained the word

we searched for.

• Very often what it returns isn’t exactly what we wanted.– Sometimes we only want on column of output– Sometimes we need to change part of its output.

3

sed

• Used to search for and replace things.

• sed s/search/replace/g

• Example:– sed s/a/b/g– Search for all occurrences of “a” and replace them

with “b”– sed s/blah//g– Search for all occurrences of “blah” and remove it…

[replace it with nothing].

4

Searching and replacing

• Lets imagine you run some command such as

• ls

• Lots of your files end in .txt, but you don’t want to show that on the screen.

• ls | sed s/.txt//g

5

Other Example

• Look through a file and replace all occurrences of the word “edna” with “enda”

• cat file.txt | sed s/edna/enda/g > newfile.txt

• Note that sed doesn’t actually change the file, it just outputs the new file, which you can then redirect.

6

Exercise

• Write a script which takes 3 arguments:replace myfile.txt a b

• This should run through the file myfile.txt and replace all a’s with b’s. It should output the result to the screen.

7

Regular Expressions

• Sed actually supports an entire language called regular expressions. They allow you to do many complex things, however they are beyond the scope of this class.

• For more information– man regex

8

Information in Columns

• Sometimes the text you want is only one column of the output.

• For example– w

• Gives you way too much information. What if you only care about who is logged in.– You can use awk to find out

9

awk

• awk is an entire programming language.

• It was specifically designed to help you parse lots of text and do stuff with it.

• For this class we are just going to use it to show us one column of text at a time.

10

Using awk

• awk ‘{print $1}’– The $1 here has nothing to do with shell parameters.– This would mean print out only the first column of the

input.

• awk ‘{print $2}’– This would print out the 2nd column.

• etc.

11

Who’s on

• If the only thing we care about is WHO is logged in, we can do:

• w | awk ‘{print $1}’ | sort | uniq | more

• This would give us a sorted list of every person logged in.

12

One Column

• Imagine you have a grade file, and you do:

• grep -i hw grades

HW1: 80

HW2: 90

HW3: 20

• However you just want the actual grades.

• grep -i hw grades | awk ‘{print $2}’

• Will print out only the second column.

13

More than one column

• awk can print more than one column at a time

• Perhaps you want file name, and size:

• ls -l | awk ‘{print $9 “ - “ $5 }’

14

Exercise

• Write a script which:– Takes one argument, (a users name)– Uses the last command to find out when that user last

logged in.– However it should one line, like this:

enda last logged in Mar 18 at 23:55

15

last on

last enda | awk '{print $1 " last logged in " $5 $6 " at " $7}' | head -1– This does a last on enda.– Looks for the first, fifth, sixth, and seventh column.– Only shows us the most recent line