a gentler introduction to unix

A Gentler Introduction to Unix

Zhizhang Shen ∗

Dept. of Computer Science and Technology

Plymouth State University

April 26, 2016

Abstract

Since this course is to run C/C++ in a Unix like environment, we start with abrief introduction to Unix, based on [1], to learn a bit about some of the basic Unix

commands, the directory structure of the Unix system, and how to compile and run aC program consisting of either one file or multiple files.

We will then proceed to learn now to write programs in C, and C++ to a lesserdegree, to solve various problems, and then go through the exact process as mentionedabove to compile and run such programs.

Later on, we will also look at a variety of Unix system calls and see a comprehensiveexample as how C is used to write programs that make use of system information as

kept by Unix.

Contents

1 An introduction 2

1.1 How to log into the system? . . . . . . . . . . . . . . . . . . . . . . . . . . . 31.2 Simple Unix commands . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3

2 Unix file system 4

2.1 Names of files and directories . . . . . . . . . . . . . . . . . . . . . . . . . . 52.2 Work with files . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 72.3 Work with directories . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 102.4 Background processing . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 132.5 Miscellaneous commands . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 13

∗Address correspondence to Dr. Zhizhang Shen, Dept. of Computer Science and Technology, Plymouth

State University, Plymouth, NH 03264, USA. Email address: [email protected].

1

3 A Unix shell 14

3.1 How does a shell work? . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 143.2 Options and arguments of commands . . . . . . . . . . . . . . . . . . . . . . 153.3 Redirection . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 163.4 Group commands . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 163.5 Pipes and tees . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 163.6 Quoting special characters . . . . . . . . . . . . . . . . . . . . . . . . . . . . 17

4 The editors 18

5 C programming in Unix 19

5.1 First thing first . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 195.2 Are you bored with it? . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 195.3 Speak mathematically . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 205.4 Cut it into pieces . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 20

6 A debugger 22

6.1 Set break points . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 236.2 How to proceed from a break point? . . . . . . . . . . . . . . . . . . . . . . . 246.3 Check out the values . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 256.4 A final example . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 26

7 Shell scripts 29

7.1 Positional variables . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 317.2 User-defined variables . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 33

8 Unix system interface 35

8.1 Look at one call in details . . . . . . . . . . . . . . . . . . . . . . . . . . . . 358.1.1 An example . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 368.1.2 Another example . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 378.1.3 Scripts again? . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 38

8.2 An example: Listing directories . . . . . . . . . . . . . . . . . . . . . . . . . 418.2.1 Given a standard interface... . . . . . . . . . . . . . . . . . . . . . . . 428.2.2 ... and an implementation of the interface . . . . . . . . . . . . . . . 458.2.3 A little summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 46

1 An introduction

Unix is a popular, and powerful operating system. It supports multitasking and time-sharing (Think about how Turing is operating.). It consists of a kernel, , a file system, ashell and various utilities. As the core of the system, the kernel controls everything inside

2

the computer and manages all kinds of resources, in particular, it implements various queuesto support the multitasking ability.

With the support of the file system, Unix organizes all kinds of data into a bunch of files,categorized into directories (Referred to as folders in Windows platform).

The Unix shell interprets user commands and passes their requests to the kernel so thatthey can be further carried out.

A utility is just a useful piece of software that makes the system more user friendly. Forexample, we can print out a calendar, list all the files out in various forms, etc..

There are various versions of Unix, such as System V Unix, BSD Unix, and Unix-likesystems, such as Linux. Turing is a Linux based system. Even the Apple iOS is built on topof Unix.

Since Unix is a multiuser system, we need a user account/password to get into the system,which is just the usual myPlymouth account.

1.1 How to log into the system?

We can get into Turing via Secure Shell (SSH), under All Programs/Network if you use aprogramming lab on campus.

If you prefer to remotely log into Turing, you need to install a remote login program,e.g., SSH, which can be obtained via https://shareware.unc.edu/. If you do a Google on“ssh putty”, you can also find and download another one of this nature, PuTTY. There is aMAC version, as well.

1.2 Simple Unix commands

Once getting into the system, we can immediately try out a few simple things.

• Change your password.

% passwd

Changing password for user zshen.

Current password: 1234567

New password: 7654321

Re-enter password: 7654321

passwd: all authentication tokens updated successfully.

%

Here, by ‘%’, we mean the command line prompt, or simply prompt. It is often associatedwith a user path, which we will explain further later on. For example, in my case, theactual prompt looks like the following:

/home/zshen >

3

Question: What is the path in your case?

• What is the date today?

% date

Mon Dec 21 08:59:39 EST 2015

• Who am I?

% who am i

zshen pts/0 2015-12-21 08:59 (d-24-233-100-151.cpe.metrocast.net)

• What is it?

% man date

DATE(1) User Commands

With Unix, you can get access to the Unix manual with the man command.

• I am done.

% logout

Lab work 1.2: Unix commands are case sensitive. Try the following out with Turing tosee what happens: WHO, CAL 2016, DATE, Who Am I. You might also want to use themanual command 1 man to check if they are validate Unix commands, e.g.,

% man WHO

Collect what you have done with the above in a ‘.txt’ file and send it in as part of yourreport for this lab. (Check out the course page for the whole assignment for Lab 1.)

2 Unix file system

There is nothing special, compared with what we have been dealing with every day with theWindows system, except perhaps a folder in Windows is referred to as a directory in Unix.

To refresh your memory, an ordinary file can be either a text file or a binary file. A textfile is usually represented as ASCII codes; and a binary file, on the other hand, is composed asa sequence of binary bits, intended for a computer to use. A special file contains informationnecessary to work with a certain device.

Structurally speaking, a bunch of related files are organized into a directory, and a bunchof directories are organized into a parent directory, ..., until everything is collected under a

1Notice that you can exit the manual mode by entering ‘q’.

4

root directory. From an operating system’s perspective, a directory is also regarded as a file.Thus, the whole thing is organized as a tree, which you should know very well after takingCS2381 Data Structure and Intermediate Programming.

The very reason to use a tree structure to organize all these directories and files isthat, as you should have learned in either MA2200 Finite mathematics or MA3200 Discretemathematics, in such a structure, the path between any two nodes, especially that betweenthe root directory and any other directory and/or file, is unique. This simple fact allows usto use the same name to identify different entities in such a structure.

At the top level, a Unix file tree contains the following directories: bin, containingthe software for the shell and most of other commands; dev, containing all the specialfiles associated with various devices; etc, containing various administrative files, such asthe list of the authorized users; home, containing the home directories of various users;tmp, containing you-know-what; usr, containing something useful; and var, containingsomething that varies, e.g., users’ mailboxes.

When just logging into the system, you are in your working directory. For example, whenI log into the system, I am in my working directory /home/PLYMOUTH/zshen. You can alwaysfind out your current working directory by using the print working directory command pwd.Once in, you can change to any directory you want to work with, by using the directory changecommand, cd. We will explain how to change to the desired directory after understandinghow to identify the location of such directories in the following Section 2.1.

2.1 Names of files and directories

Every file and directory is identified by a name. On most of the systems, a name containsbetween 1 and 255 characters out of the following: upper case letters, lowercase letters,digits, period, underscore and comma. You should avoid some of the special symbols suchas ‘&’, ‘*’, ‘\’, various brackets, ‘%’, etc., when coming up with a name, and you should notuse various reserved words as the names of your directories/files.

To refer to a file located in your current directory, you only need to use its name. But,if you want to use a file located in another directory, you have to use its pathname, whichuniquely identifies its location in the whole file structure.

For example, the root directory is ‘/’, and the absolute pathnames for the root’s childdirectories, as shown in Figure 1 is ‘/bin’, ‘/dev’, etc..

In general, to form the absolute pathname of a node n in a tree structure, we start with‘/’, the name of the root, and collect the names of all the nodes along the unique path fromthe root to n in a sequence, and use ‘/’ to separate all such names.

For example, as shown in Figure 1, the directory ‘home’ contains two sub-directories: jackand jill. Their absolute path names are ‘/home/jack’ and ‘/home/jill’, respectively. Sim-ilarly, the absolute pathname for the file kangaroo is ‘/home/jill/Marsupials/kangaroo’.

Although the absolute path name uniquely identifies a file, and allows us to give the samename to different files, it is certainly an overkill. Moreover, we usually don’t have accessrights to some of these directories, especially, the root directory. Thus, we almost always

5

Figure 1: An example of a Unix tree

use relative pathname to refer to a certain node in a file structure. This notion is also basedon the key property of a tree: there is a unique path between any two nodes, going throughthe root of the minimum tree which includes both nodes.

For example, if we are in the directory jack, and want to refer to the directory Marsupials,we notice home is the root of the minimum tree that contains both of these two nodes. Wethus 1) move up to home, the parent of jack, as represented with ..; 2) then move down tojill; and 3) move further down to Marsupials, while using ‘/’ to separate all these threenames. As a result, the name of Marsupials relative to jack is “../jill/Marsupials”.

We can also use ‘.’ to refer to the current working directory, as in, e.g., ‘./Oceans’,assuming the current working directory is still “jack”. But we prefer to use “Oceans” as itsaves us two key strokes. On the other hand, because of an access privilege issue, when wewant to execute a program in the current working directory, e.g., a.out, we have to say

% ./a.out

Lab work 2.1:

1. Which of the following are valid names for ordinary Unix files, and why?

foo, guess?, book.chap1, BOOK.chap2, 2good2Btrue, {2bad}, rank*, serial#

2. Which of the following are valid names for directories, and why?

Dir2, Directory.3, *Hook, |Line1|, "Sinker", money.$, .hideNseek, 777

3. In Figure 1, what are the absolute path names for root, bin, jill, and kangaroo,respectively?

6

4. Again, in Figure 1, suppose that Marsupials is now the working directory, what arethe relative pathnames of root, bin, jill, and kangaroo, respectively?

5. In Figure 1, the directory jack has two subdirectories, Continents and Oceans, whatare their absolute pathnames?

Collect what you have done in a ‘.txt’ file and send it in as part of your report for this lab.(Check out the course page for the whole assignment for Lab 1.)

2.2 Work with files

Once a directory is identified, we can list all the files included in that directory using the ls

command.

1. List all the files in the current directory 2

% ls

/home/zshen > ls

a.out first.c hello lucky second.c

basicExamples format.c hello.c pow.c sqroot.c

char formatFirst.c longestLine power.c temprature

counting getline.c lsOutput printd.c

2. A hidden file, i.e., a file with its name starting with a ‘.’, will not be listed, unless weuse the list all command.

% ls -a

. char formatFirst.c longestLine power.c temprature

.. counting getline.c lsOutput printd.c

a.out first.c hello lucky second.c

basicExamples format.c hello.c pow.c sqroot.c

3. We can have a look at the content of a text file by using more:

% more first.c

#include <stdio.h>

main(){

printf("I feel lucky today

\n");

}

2What you will get is certainly different from mine.

7

4. There are four ways to create a file: You can either 1) redirect the “standard input”into a file; 2) copy an existing file; 3) use a text editor; or 4) use a computer programto generate a file.

• If you don’t have any files to play with, the easiest way to create some files is toredirect the output of the standard input into a file with ‘>’, as follows :

% ls

output.txt

% ls > sampleRedirect.txt

% more sampleRedirect.txt

output.txt

sampleRedirect.txt

• You can also copy an existing file with the cp command.

% ls

output.txt sampleRedirect.txt

% cp output.txt anotherOutput.txt

% ls

anotherOutput.txt output.txt sampleRedirect.txt

• If you want to redirect the output of the standard input and append it to anexisting file, you can use the ‘>>’ operator.

% more output.txt

This is a test

% ls


% ls >> output.txt

% more output.txt

This is a test

anotherOutput.txt

output.txt

sampleRedirect.txt

• We will talk about how to use an editor to create a text file in Section 4; and nowto create a file with a program in Section 8.

5. We can change the name of a file, or even move a file to another place, with the movecommand ‘mv’.

% ls


8

% mv output.txt result.txt

% ls

anotherOutput.txt result.txt sampleRedirect.txt

You can also move a file somewhere else by changing its absolute path name. Here thecommand mkdir creates a directory sampleDir, which we will discuss in Section 2.3.

% mkdir sampleDir

% ls

anotherOutput.txt result.txt sampleDir sampleRedirect.txt

The following moves the file result.txt into the just created directory sampleDir.

% mv result.txt sampleDir

% ls sampleDir

result.txt

You should use the man(ual) command to check out the other formats, option, etc. ofthis move command.

6. To delete a file, you use rm as follows:

% ls

anotherOutput.txt sampleDir sampleRedirect.txt

% mv sampleDir/result.txt result.txt

% ls


% cp result.txt output.txt

% ls

anotherOutput.txt output.txt result.txt sampleDir sampleRedirect.txt

% rm output.txt

% ls


7. With ls, the ‘l’ options shows all the information of the items in a directory.

% ls -l

total 16

-rw-r--r-- 1 zshen domainûsers 64 Jan 28 08:23 anotherOutput.txt

-rw-r--r-- 1 zshen domainûsers 64 Jan 28 08:16 result.txt

drwxr-xr-x 2 zshen domainûsers 4096 Jan 28 08:30 sampleDir

-rw-r--r-- 1 zshen domainûsers 30 Jan 28 08:22 sampleRedirect.txt

9

8. Look at the aboove long listing, you might be wondering what does “rw-r--r--” mean.They deal with the access privilege of a file or a directory in three groups: owner, groupand public. For each group, it tells its access privileges regarding whether it can beread, written or executed. Thus, “rw-r--r--” means that the owner can read andupdate, but cannot execute, this file; all the members of his/her group, as well as thepublic, can only read this file.

9. Only the owner of a file, or a directory, can change such privileges with the chmod

command. For example,

% more result.txt

This is a test

anotherOutput.txt

output.txt

sampleRedirect.txt

% ls -l result.txt

-rw-r--r-- 1 zshen domain^users 64 Jan 28 08:16 result.txt

% chmod u-r result.txt

% more result.txt

result.txt: Permission denied

Besides ‘u’ for owner, ‘a’ for all, you can also use ‘o’ for others (public) and ‘g’ forpublic. Moreover, besides ‘=’ for assignment, and ‘-’ for remove, you can also use‘+’ for add a permission. For example, the following gives everyone the read accessprivilege to the file.

% chmod a=r result.txt

% ls -l result.txt

-r--r--r-- 1 zshen domain^users 64 Jan 28 08:16 result.txt

2.3 Work with directories

It is quite common to develop a tree like directory structure to manage the files more effi-ciently by using various directory related commands.

1. When working with directory, the most important thing is to know where you are,i.e., what is your working directory, with the pwd command.

% pwd

/home/zshen/testDir/parent

% ls


10

2. To create another directory within the current directory, you use mkdir.

% mkdir anotherDir

% ls

anotherDir anotherOutput.txt result.txt sampleDir sampleRedirect.txt

3. Change the working directory to this newly created directory

% cd anotherDir

% ls

4. Come back to the previous working directory

% pwd

/home/zshen/testDir/parent/anotherDir

Remember “..” stands for the parent directory of the current working directory.

% cd ..

% pwd

/home/zshen/testDir/parent

% ls

anotherDir anotherOutput.txt result.txt sampleDir sampleRedirect.txt

5. To remove a directory, we use rmdir

% rmdir anotherDir

% ls


Lab work 2.3:

1. Create the directory structure as shown in Figure 1, notice that kangaroo is a file, butnot a directory. Use “root” as the name of the root directory.

2. Assume that we will set up some additional subdirectories for jack as follows: Continentsnow holds Africa, Antarctica, Asia, Austria, Europe, NAmerica, and SAmerica;

and each of these directories holds some countries and regions of their own.

(a) Use Unix command to set up the above structure.

(b) Give the absolute path names for the following countries: Norway, India, Egyptand Argentina

11

(c) Assume that your current working directory is USA, accomplish the followingusing a single command line with relative path names.

i. List the content of Marsupials.

ii. List the content of Australia.

iii. Make a copy of the kangaroo file and place this file, also named kangaroo,

in the Australia directory under Continents.

(d) Repeat the above using absolute path names.

3. The echo command takes a line of input that you type in through the keyboard andkicks it right back on the screen. Try it out

% echo try this out.

try this out.

Create a file fun by redirecting the output with the echo command, with the help ofthe redirection mechanism (Cf. The first item in Page 8).

4. Use the commands who, who am i, and date to append more data to the fun file.

5. Come up with a file stuff and place it in your working directory. Then do the following:

(a) Give everyone permission to read stuff, but do not change any other accessprivilege.

(b) Permit the owner and group members to read and write the file; remove theseprivileges from everyone else.

(c) Give the owner and group members permission to execute stuff while giving theowner sole permission to read or write it.

6. A hidden file has a name starting with ‘.’. Use the cal utility and the redirectionoperator ‘>’ to create a .hidden file. How to show the content of this file?

7. Create a new directory Misc and move the file fun into Misc.

8. Without leaving your current directory, create a directory Vacations under Misc.

Then create calendars for the summer months of 2016 and move them into the Vacationsdirectory.

9. Redo Lab work 2.1.1 and 2.1.2 by testing them out, using appropriate commands. Tellme what happens when you try to create, e.g., {2bad}, etc..

Collect what you have done in a ‘.txt’ file and send it in as your lab report.

12

2.4 Background processing

As we mentioned earlier, Unix is a powerful OS that supports multitasking in terms ofmultiple processes identified with PID. Since there is only one processor in the machine, atany time, one process is running, while the others are simply waiting. (We will talk a lotmore details along this line in CS4310, i.e., the OS course.)

The command ps will show all the process id that have been running.

% ps

PID TTY TIME CMD

3384 pts/0 00:00:00 bash

6611 pts/0 00:00:00 ps

We can use the kill command to terminate a process, stop to suspend a process, bgbrings it to the background, and fg to bring a process to the foreground. The ‘&’ at the endof a line puts it into the background.

/home/zshen > (sleep 60; echo stop this command) &

[1] 6614

/home/zshen > ps

PID TTY TIME CMD

3384 pts/0 00:00:00 bash

6614 pts/0 00:00:00 bash

6615 pts/0 00:00:00 sleep

6616 pts/0 00:00:00 ps

/home/zshen > kill 6615

/home/zshen > -bash: line 45: 6615 Terminated sleep 60

stop this command

2.5 Miscellaneous commands

There are also quite a few other commands, such as comm, crypt, cut, diff, etc.. Tocheck them out for details, you can use the manual command man to find out their respectivesyntax and semantics. For example,

% man grep

GREP(1) GREP(1)

NAME

grep, egrep, fgrep - print lines matching a pattern

SYNOPSIS

13

grep [options] PATTERN [FILE...]

grep [options] [-e PATTERN | -f FILE] [FILE...]

DESCRIPTION

Grep searches the named input FILEs (or standard input if no files are

named, or the file name - is given) for lines containing a match to

the given PATTERN. By default, grep prints the matching lines.

In addition, two variant programs egrep and fgrep are available.

Egrep is the same as grep -E. Fgrep is the same as grep -F.

OPTIONS

-A NUM, --after-context=NUM

Print NUM lines of trailing context after matching lines.

Places a line containing -- between contiguous groups of

matches.

Below shows an application. We will see how this utility does it later on.

/home/zshen > more first.c

#include <stdio.h>

main(){

printf("I feel lucky today\n");

}

/home/zshen > grep printf first.c

printf("I feel lucky today\n");

3 A Unix shell

As we mentioned earlier that a Unix shell interprets and executes user commands.

3.1 How does a shell work?

The general process looks like the following:

1. The shell displays a prompt such as ‘% ’.

%

2. You type in a command such as cal.

14

% cal

3. The shell interprets your command by looking for an appropriate software. If it can’tfind it, it will print out an error message.

% wrong

-bash: wrong: command not found

4. If it finds the command, the kernel runs the requested software to give you the output.

% cal 2 2016

February 2016

Su Mo Tu We Th Fr Sa

1 2 3 4 5 6

7 8 9 10 11 12 13

14 15 16 17 18 19 20

21 22 23 24 25 26 27

28 29

5. The shell then prints another prompt and waits for the next command.

%

3.2 Options and arguments of commands

For the cal command, you can specify which month and year you want. Here both themonth and the year are the arguments of this command.

Sometimes, we have options for a command. For example, we have -a, -F, -l, -r,

-s, -t, and -u for the listing command ls, where, e.g., the -F option classifies all the objectswithin a directory by showing a ‘/’ after each directory and a ‘*’ after each executable file.

% ls

anotherDir anotherOutput.txt a.out hello.c result.txt sampleDir

% ls -F

anotherDir/ anotherOutput.txt a.out* hello.c result.txt sampleDir/

%

Lab work 3.2:

1. What does “-F” stand for? Find it out and do the same for all the other options.

2. Try out all the above options for the ls command.

15

3.3 Redirection

We already talked about this stuff earlier, e.g.,

% more result.txt

This is a test

anotherOutput.txt

output.txt

sampleRedirect.txt

% wc < result.txt > wcOutput.txt

% more wcOutput.txt

5 7 64

%

Here wc is another utility that “print the number of newlines, words, and bytes in files”.

Lab work 3.3:

1. Use man to dig out all the details of this wc utility.

2. Run wc to find out the number of characters in the calendars for all the summer months,i.e., June through August, of 2016 put together.

3.4 Group commands

Besides typing one command at a time, we can also type in several at a time and place themin the same line, which will then be executed sequentially. For example,

% who am i; ls; cal 2 2016

zshen pts/0 2015-12-21 08:59 (d-24-233-100-151.cpe.metrocast.net)

anotherOutput.txt a.out daily fun I_did_it noomment.c output.txt programs

February 2016


1 2 3 4 5 6

7 8 9 10 11 12 13

14 15 16 17 18 19 20

21 22 23 24 25 26 27

28 29

3.5 Pipes and tees

The pipe command, ‘|’, sends the output of one command as the input of another command.For example,

16

% ls | more

input

output

output.txt

sampleRedirect

Here, the command ls throws its output, the content of the working directory, into a tem-porary file, which is passed as input to more. The latter command then displays such acontent.

On the other hand, “|tee” does one more thing: besides passing the output of onecommand to another as input, it also saves the output in a file. In the following, besidespassing the output of ls to more to display, it also saves such a content into the file IsOut.

% ls |tee lsOut |more

input

output

output.txt

sampleRedirect

% ls

input lsOut output output.txt sampleRedirect

% more lsOut

input

output

output.txt

sampleRedirect

Lab work 3.5:

1. Use “|tee” to view the calendar for 2016, and also save them in a separate file.

3.6 Quoting special characters

A file name should not contain any of the special characters that Unix uses for some specificpurpose. But, when we have a need to use any of the special characters, we have to tellthe system using the escape character ‘\’. For example, ‘*’ stands for a wild card, i.e.,everything, while “” stands for itself.

% echo *

anotherDir anotherOutput.txt a.out hello.c result.txt sampleDir wcOutput.txt

% echo \*

*

We use grave accent, ‘‘’, i.e., the key above the “Tab” key in a standard keyboard 3, toindicate a command that we want the shell to run. For example,

3In the standard ASCII code chart, the binary code, Octadecimal, Decimal, and Hexadecimal code of

grave accent is 110 0000, 140, 96 and 60, respectively.

17

% date

Mon Dec 21 09:09:02 EST 2015

Nothing leads to nothing.

% echo It is not date

It is not date

If we use ‘’’, apostrophe (Decimal code 39), we get the following:

% echo It is not ’date’

It is not date

We will get the following if we use the grave accent.

% echo It is not ‘date‘

It is not Mon Dec 21 09:09:30 EST 2015

%

Lab work 3.6:

1. What does each of the following do, and why?

echo *

echo /*

echo \*

echo "\*"

echo

echo */*

2. Investigate the wild cards such as ‘*’, ‘?’, and use cat, another utility, to show thecontents of all the files with their names ending in “ing”; and how to list all the fileswith their names containing either ‘x’ or ‘X’.

4 The editors

There are a few editors that we can use to enter a program, such as vi, pico and emacs.

We once all used pico to edit emails. It is pretty easy to use.

% pico newFile

Try it out yourself.A very popular one is called vi, which is also always available with Unix. The negative

side is that it is not as easy. Check out the online vi manual.

Lab work 4: Make sure that you know at least one of the two editors reasonably well,before proceeding to the next unit.

18

5 C programming in Unix

5.1 First thing first

It is pretty easy to compile a c program in Unix, using the cc, or gcc, facility. For example,

% more hello.c

/* the first program in c */

#include <stdio.h>

int main(void){

printf("Hello, world\n");

return 0;

}

% gcc hello.c

% ./a.out

Hello, world

5.2 Are you bored with it?

You don’t need to always call the executable a.out, for example

% gcc hello.c -o hello

% ./hello

I feel lucky today

Here is another example which calculates the reciprocal of an integer.

% more recip.c

/* Compute reciprocals */

#include <stdio.h>

int main(void){

int n;

float recip;

printf("This is a program computes reciprocals.\n");

printf("Enter a number: ");

scanf("%d", &n);

recip=1/(float) n;

printf("The reciprocal of %d is %f.\n", n, recip);

return 0;

}

19

% gcc recip.c -o recip

% ./recip

This is a program computes reciprocals.

Enter a number: 34

The reciprocal of 34 is 0.029412.

5.3 Speak mathematically

When using a mathematical function, we have to add on the -lm option to link to themathematics library. For example,

% more sqroot.c

/* Compute square roots */

#include <stdio.h>

#include <math.h>

int main(void){

float n, x, root;

printf("This program computes square root.\n");

printf("Enter a number: ");

//printf("%3.1f", sqrt(32));

scanf("%f", &n);

x=n;

printf("The number you entered is %f: ", x);

printf("The square root of %f is %f.\n", n, sqrt(n));

return 0;

}

% gcc sqroot.c -lm

% ./a.out

This program computes square root.

Enter a number: 34

The number you entered is 34.000000: The square root of 34.000000 is 5.830952.

5.4 Cut it into pieces

We can also work with a program consisting of multiple files. For example,

% more main.c

/* Illustrate multiple source files */

void chicago(void);

20

void indiana(void);

int main(void){

chicago();

indiana();

chicago();

return 0;

}

% more chicago.c

/* The chicago() function */

#include <stdio.h>

void chicago(){

printf("\nI’m waiting at O’Hare International,\n ");

printf("Airport, a fun place");

}

% more indiana.c

/* The indiana() function */

#include <stdio.h>

void indianapolis(void);

void indiana(void){

printf("Back home again, Indiana.\n");

indianapolis();

printf("Wander Indiana--come back soon!\n ");

}

% more indy.c

/* The indianapolis() function */

#define POP2000 1.6

#include <stdio.h>

void indianapolis(void){

printf("\nWelcome to Indianapolis, Indiana.\n");

printf("Population: %f million.\n", POP2000);

}

% cc -c main.c

% cc -c chicago.c

21

% cc -c indiana.c

% cc -c indy.c

% cc *.o

% ./a.out

I’m waiting at O’Hare International,

Airport, a fun placeBack home again, Indiana.

Welcome to Indianapolis, Indiana.

Population: 1.600000 million.

Wander Indiana--come back soon!

I’m waiting at O’Hare International,

Airport, a fun place.

Lab work 5: Test out all the program samples as shown in this section, and let me knowthat “I’m done.”.

6 A debugger

A debugger is perhaps one of the most important tools for a system programmer, similarto a Java programmer. It allows us to check out various snapshots of the execution of aprogram, namely, values of various variables. It allows us to determine whether a particularline in a program will be the next one to be executed. We can also use it to execute one lineat a time, instead of letting the whole thing go through in a single shot. We can also use itto observe how a program behaves with a collection of choice points.

We will now check out this many possibilities through a particular debugger gdb thataccompanies gcc. Let’s use the following code as a running example.

1. #include <stdio.h>

2.

3. main(){

4. int i, sum;

5.

6. sum=0;

7. for(i=0; i<10; i++)

8. if(i < 5)

9. sum=sum+1;

10. else

11. sum=sum+((i-3)/2+(i/3)); //line 11

12.

13. printf("sum=%d\n", sum);

}

22

We can certainly run it as follows.

% gcc sum.c

% ./a.out

sum=24

Let’s send it through a debugger:

% gcc -g sum.c

% gdb a.out

GNU gdb (GDB) Red Hat Enterprise Linux (7.2-60.el6)

Copyright (C) 2010 Free Software Foundation, Inc.

License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>

This is free software: you are free to change and redistribute it.

There is NO WARRANTY, to the extent permitted by law. Type "show copying"

and "show warranty" for details.

This GDB was configured as "x86_64-redhat-linux-gnu".

For bug reporting instructions, please see:

<http://www.gnu.org/software/gdb/bugs/>...

Reading symbols from %a.out...done.

(gdb)

In the above, to facilitate the debugger, we use the “-g” switch to come up with addi-tional information, such as a symbol table, during the compilation process. Moreover, whenthis switch is used, no optimization is done when compiling the codes, which makes it lesscomplicated to do the debugging.

After compiling, we run the debugger on the executable a.out, which simply loads theexecutable into the debugger environment. We are now ready to execute various debuggercommands. Once any such command is done, it comes back to the debugger prompt

(gdb)

Question: How to get out of gdb?

(gdb) quit

%

6.1 Set break points

It is often useful to execute a program partially, i.e., through part of a program, which canbe done by setting a break point. It tells the debugger to run the program until that breakpoint is reached, when the program just paused. We can then do other things to the pausedprogram. For example,

23

(gdb) break 11

Breakpoint 1 at 0x4004bc: file sum.c, line 11.

(gdb) run

Starting program: /users/facstaff/S/zshen/Courses/CS2470/programs/Debugger/a.out

Breakpoint 1, main () at sum.c:11

11 sum=sum+((i-3)/2+(i/3));

(gdb)

Question: What should be value of i at this point?Answer: 5. Since if i is less than 5, it would go through the then part.

Question: How do we find it out?Answer: We can certainly check this out using another debugger command display:

(gdb) display i

1: i = 5

(gdb)

We can also check out where the program is paused.

(gdb) where

#0 main () at sum.c:11

6.2 How to proceed from a break point?

There are three ways: step, next and continue.

1. The step command executes the next line in the logic flow and then pause again. Sortof “one step at a time”.

Question: What is the next line at this point in the logic flow?

Answer: Since line 11 is the last line of the loop body, the control has to go back tothe for line, i.e., line 7.

Question: How do we find it out?

Answer: Use step.

(gdb) step

7 for(i=0; i<10; i++)

1: i = 5

Question: Then what?

24

(gdb) step

8 if(i < 5)

1: i = 6

(gdb)

The first step executes line 11, and brings the control to the loop at line 7, when thevalue of i is still 5. The next step does the increment of i and brings the control toline 8, where the value of i turns into 6.

2. The next command does the same, except if the next line is a function call, it willcomplete everything that call does.

(gdb) next


11 sum=sum+((i-3)/2+(i/3));

1: i = 6

(gdb) next

7 for(i=0; i<10; i++)

1: i = 6

(gdb) next

8 if(i < 5)

1: i = 7

The next command might be more preferable over step.

3. The command continue allows the execution continue until either a breakpoint isreached, the program exits normally, or it tries to do something illegal.

(gdb) continue

Continuing.


11 sum=sum+((i-3)/2+(i/3));

1: i = 7

6.3 Check out the values

There are two ways to observe the values of a variable: print and display.

1. The print command is a one-time request to see the value. The debugger shows itscurrent value then forgets about it.

25

(gdb) print i

$1 = 7

2. As we have already seen, the display command shows it on an ongoing basis: Thedebugger will show its value every time the program is paused.

6.4 A final example

The program sum.c generates different values of sum for different value of i. We might wantto see the big picture as what specific values of sum will be generated for what specific valuesof i.

What we could do is to set break points at lines 8 and 9, and print out values of sum andi in that context.

% gdb a.out

%GNU gdb (GDB) Red Hat Enterprise Linux (7.2-60.el6_4.1)

Copyright (C) 2010 Free Software Foundation, Inc.

License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>

This is free software: you are free to change and redistribute it.

There is NO WARRANTY, to the extent permitted by law. Type "show copying"

and "show warranty" for details.

This GDB was configured as "x86_64-redhat-linux-gnu".

For bug reporting instructions, please see:

<http://www.gnu.org/software/gdb/bugs/>...

Reading symbols from /home/PLYMOUTH/zshen/Courses/CS2470/programs/unixExamples/debug6/

a.out...done.

(gdb) break 8

Breakpoint 1 at 0x4004dd: file sum.c, line 8.

(gdb) break 9

Breakpoint 2 at 0x4004e3: file sum.c, line 9.

(gdb) run

Starting program: /home/PLYMOUTH/zshen/Courses/CS2470/programs/unixExamples/debug6/a.out


8 if(i < 5)

Missing separate debuginfos, use: debuginfo-install glibc-2.12-1.132.el6.x86_64

(gdb) display i

1: i = 0

(gdb) step


9 sum=sum+1;

1: i = 0

(gdb) display sum

26

2: sum = 0

(gdb) continue

Continuing.


8 if(i < 5)

2: sum = 1

1: i = 1

(gdb)

Continuing.


9 sum=sum+1;

2: sum = 1

1: i = 1

(gdb)

Continuing.


8 if(i < 5)

2: sum = 2

1: i = 2

(gdb)

Continuing.


9 sum=sum+1;

2: sum = 2

1: i = 2

(gdb)

Continuing.


8 if(i < 5)

2: sum = 3

1: i = 3

(gdb)

Continuing.


9 sum=sum+1;

2: sum = 3

1: i = 3

(gdb)

Continuing.

27


8 if(i < 5)

2: sum = 4

1: i = 4

(gdb)

Continuing.


9 sum=sum+1;

2: sum = 4

1: i = 4

(gdb)

Continuing.


8 if(i < 5)

2: sum = 5

1: i = 5

(gdb)

Continuing.


8 if(i < 5)

2: sum = 7

1: i = 6

(gdb)

Continuing.


8 if(i < 5)

2: sum = 10

1: i = 7

(gdb)

Continuing.


8 if(i < 5)

2: sum = 14

1: i = 8

(gdb)

Continuing.


8 if(i < 5)

28

2: sum = 18

1: i = 9

(gdb)

Continuing.

sum=24

Program exited with code 07.

(gdb) quit

%

The order of the break points does not matter, the debugger will pause whenever a breakpoint is reached.

Break points can be removed using the clear command.

(gdb) clear 9

Deleted breakpoint 2

(gdb) continue

Continuing.

As we just saw, once we are done, we simply quit.

(gdb) quit

A debugging session is active.

Inferior 1 [process 18704] will be killed.

Quit anyway? (y or n) y

%

Incidentally, this debugger also works with g++, i.e., the c++ stuff.

Lab work 6

1. Play with the debugger with the sum.c program, and familiarize with its features.

2. Find a non-trivial program in c that you did in this semester. Identify one or more as-pects that you want to identify, particularly, a malfunctioning feature of your program,i.e., something that you did wrong. Then use various features of this gdb debugger toidentify the bugs, and come up with a correct program.

7 Shell scripts

A very powerful feature of Unix is that we can group together various Unix commands tocome up with a script, or a little program, that will execute. For example, the followingscript, simpleShell, containing the following lines:

29

% more simpleShell

#A simple shell script

cal

date

who

Here ‘#’ indicates a comment. We can then run the program by using a redirection:

% sh < simpleShell

April 2016


1 2

3 4 5 6 7 8 9

10 11 12 13 14 15 16

17 18 19 20 21 22 23

24 25 26 27 28 29 30

Fri Apr 22 10:41:48 EDT 2016

zshen pts/0 2016-04-22 10:39 (cszs04.plymouth.edu)

%

We can also make this file executable by changing its access privilege using chmod, whichwas discussed in page 10:

% chmod u+x simpleShell

% ./simpleShell

April 2016


1 2

3 4 5 6 7 8 9

10 11 12 13 14 15 16

17 18 19 20 21 22 23

24 25 26 27 28 29 30

Fri Apr 22 10:43:15 EDT 2016

zshen pts/0 2016-04-22 10:39 (cszs04.plymouth.edu)

%

In this sense, shell becomes a system programming language, where the control structuresare pretty much the same stuff; but the variables are different: besides the user-definedvariables, there are two other kinds: environment variables and positional variables.

Environment variables are used to keep values for special shell variables such as HOME,

TERM, referring to your home directory, the type of the terminal you use, respectively. Theyset the scene for everything, but we will not touch on them too much here. We simply don’twant to mess them up.

30

7.1 Positional variables

Positional variables are used to capture the command-line arguments used by the shell script,numbered 0, 1, 2, ..., 9. Similar to the argv[] usage, $0 keeps the name if this very program,S1 the first argument, etc.. For example, the following is the echo.args script.

% more echo.args

#!/bin/sh

#Illustrate the use of positional variables

echo $0 $1 $2 $3 $4 $5 $6 $7 $8 $9

Notice that the very first line

#!/bin/sh

specifies that we will always use a particular shell, sh, to run the scripts. Thus,

% ./echo.args I love Spring!

./echo.args I love Spring!

Positional variables $1 through $9 are also collected as $*. Their number is denoted as$#. For example,

% more echoRead.args

#!/bin/sh

#Illustrate the use of positional parameters, used defined variables and

#the read command.

echo ’What is your name?’

read name

echo "Well, $name, you typed $# arguments:"

echo $0 $*

Thus,

% ./echoRead.args This is great!

What is your name?

Zhizhang

Well, Zhizhang, you typed 3 arguments:

./echoRead.args This is great!

You can also set up those variables using the set command, which is very useful. Forexample,

% more weeklyReminder

#!/bin/sh

# A daily reminder service

set ‘date‘

31

echo "Remember for today:"

case $1 in

Mon) echo "Plan the week.";;

Tue) echo "Take clothes to the cleaners.";;

Wed) echo "Attend group meeting.";;

Thu) echo "Make plans for the weekend.";

echo "Pick up clothes at the cleaners.";;

Fri) echo "Answer e-mail";;

Sat) echo "You should not be here working.";

echo "Finish your work and log off.";;

Sun) echo "Call Grandma and Grandpa.";;

esac

Let’s try to run this piece out....

% chmod u+x weeklyReminder

% ./weeklyReminder

Remember for today:

Answer e-mail

% date

Fri Apr 22 10:45:01 EDT 2016

%

Yes, we answer emails on Friday! ,

Just one more example, setdate,

% more setDate

#!/bin/sh

#Demonstrate the set command

set ‘date‘

echo "Time: $4 $5"

echo "Day: $1"

echo "Date: $3 $2 $6"

Note that when you type this ‘date‘ stuff, you have to use the grave accent, i.e., thekey above the Tab key and to the left of 1.

% chmod u+x setDate

% ./setDate

Time: 10:49:33 EDT

Day: Fri

Date: 22 Apr 2016

%

32

Question: What is going on?

Answer: The grace accented ‘date‘ run the date command, which sends back the following

Fri Apr 22 10:49:33 EDT 2016

as values of positional parameters: $1, $2, $3, $4, $5 and $6.

Lab work 7.1:

1. Play with all the scripts and really understand them.

2. Write a script to come up with a daily reminder, e.g., for the week days,

(a) If it is about 6 a.m., get up

(b) If it is about 7 a.m., have breakfast

(c) If it is about 8 a.m., start the first class

(d) If it is about 10 p.m., time to do homework

If it is either Saturday or Sunday, do something appropriate for the weekend, e.g.,cleaning the backyard.

Note: For the “about” part, you might put in a difference, say, “ about 6 a.m.” is thesegment between 5:55 and 6:05 in the morning. You might also use a wild card, e.g.,“05:5*” stands for a period between 5:50 through 5:59. Do some research to find outmore relevant expressions.

In either case, you certainly need to investigate the syntax for the conditional structureas used in shell.

7.2 User-defined variables

The following mySpell script provides a speller, with a file name as its only argument, whichis passed to a user-defined variable, file. Notice how it is declared, and later used.

% more mySpell

#!/bin/sh

# An improved spelling checker

file=$1

for word in ‘spell $file‘

do

# grep tries to find the line where $word occurs in $file

line=‘grep -n $word $file‘

#print out a line

echo " "

33

#print out the following line

echo "Missplled word: $word"

#print out the line where the misspelled word occurs

echo "$line"

done

% more testWord

this is intendly spelleed wrong

Let’s put in another wroong word

How about annother one?

% ./mySpell < testWord

Missplled word: annother

Missplled word: intendly

Missplled word: spelleed

Missplled word: wroong

Please don’t try the following delFile, if you don’t want to accept the consequence.

% more delFile

#!/bin/sh

#Delete a file interactively

filename=$1

if test ! -f $filename

then

echo "There is no file \"$filename\"."

else

echo "Do you want to delete \"$filename\"?"

read choice

if test $choice = y

then

rm $filename

echo \"$filename\" deleted

else

echo \"$filename\" not deleted.

34

fi

fi

% ./delFile fakedir

There is no file "fakedir".

% ls testFile

testFile

% ./delFile testFile

Do you want to delete "testFile"?

y

"testFile" deleted

% ls testFile

ls: cannot access testFile: No such file or directory

Lab work 7.2:

1. Modify the mySpell script so that if you fail to enter the file name, it will print areminder, e.g., “Usage: mySpell filename”.

2. Modify the delFile script so that it will test if the one to be deleted is a directory, inthis case, it should call rmdir instead of rm to delete it.

Notice that the following line tests if name is a directory, which sends back True is thevalue of $dirname holds a directory’s name.

if test ! -d $dirname

8 Unix system interface

The Unix provides its services via a set of system calls, which can be used by various userprograms. We now look at a few important and useful system calls. Since most of the Unixare written in C, such system calls also provide a way to us to have an insight look of C.

8.1 Look at one call in details

We will experiment with an important Unix system call, fork(), which creates a newprocess. When a process creates another child process, it simply creates a copy of the callingprocess (parent process), except that the child process has its own identification number, pid,and its own pointers to shared kernel entities 4. After fork() is called, two processes willexecute the next statement after this fork() call with their respective 5 data space.

4On the other hand, the system call vfork() creates a child process that shares its data space with its

parent.5This is not true with the vfork() call, which we will work with in CS4310 Operating Systems.

35

If the fork() call succeeds in the parent process, it returns the pid of the newly createdchild process. For the child process, fork() returns a 0. This tells apart the execution oftwo processes, as we will see in the later demos.

It may seem like an overkill by creating another process to carry out certain computation,but it is actually essential from at least a reliability consideration: If the newly created childprocess does anything wrong, it might crash; while the parent process will still run normally.

This technique is used extensively. For example, after the system is booted, a process 0is created, which is the ancestor of every process created during that session. Whenever acommand is to be executed; or whenever a user is to log on, a new process is created. Forthose who have taken, or are taking, CS3221 Algorithms, we discussed its usage in spawningprocesses to potentially take care of parallel tasks. This important tool will be discussed inmuch more details throw projects CS4310 Operating Systems.

You are also expected to use this technique when taking on the final project on shell forthis course.

8.1.1 An example

Below is a simple example, fork.c, involving the fork() call.

/* Slight modification of Richard Stevens example from "Advanced Programming

in the UNIX Environment"

*/

#include <stdio.h>

#include <sys/types.h>

#include <unistd.h>

#include <stdlib.h>

int globalVar = 6; /* external variable in the initialized data */

char buf[] = "Write to the standard output\n";

int main(void){

int localVar; /* local variable in the initialized data */

pid_t pid;

localVar = 88;

if (write(STDOUT_FILENO, buf, sizeof(buf)-1) != sizeof(buf)-1){

printf("write error");

exit(1);

}

printf("Now, the fork starts\n");

36

if ( (pid = fork()) < 0){

printf("fork error\n");

exit(1);

}

else if (pid == 0) {

globalVar++;

localVar++;

}

else sleep(2);

printf("pid = %d, globalVar = %d, localVar = %d\n", getpid(), globalVar, localVar);

exit(0);

}

Labwork 8.1.1: Run the program, then check it out carefully so that you can answer thefollowing questions:

1. What are the process ids for the child and the parent processes?

2. Which parts are done by both processes, by the child only, and by the parent only?

3. Why does only the child process increment the two variables, i.e., globalVar andlocalVar? Why are both of them affected?

4. Which process(es), child and/or parent, execute(s) the sleep command? Why is that?

8.1.2 Another example

More realistically, a parent wants to create a process to get something done, and will actuallywait until the newly created process to complete.

This is quite similar to the final project, where the parent process, might want to forkoff a child process to do, e.g., the ls command, and will wait there, until it is done.

The following code, consisting of two files, demonstrates this situation.

/* The parent.c file */

#include <stdio.h>

#include <stdlib.h>

#include <sys/wait.h>

int main(){

if(fork()==0){

execve("child", NULL, NULL);

exit(0);

}

printf("Process[%d]: Parent in execution...\n", getpid());

37

sleep(2);

if(wait(NULL)>0)

printf("Process[%d]: Parent detects terminating child \n", getpid());

printf("Process[%d]: parent terminating...\n", getpid());

}

/* The child.c file */

#include <stdio.h>

int main(){

printf("Process[%d]: child in execution...\n", getpid());

sleep(2);

printf("Process[%d]: child terminating...\n", getpid());

}

Notice that we have to name the executable file associated with child.c as child.

% cc child.c -o child

We can then compile and run parent.c and get the following:

% cc parent.c

% ./a.out

Process[9715]: Parent in execution...

Process[9716]: child in execution...

Process[9716]: child terminating...

Process[9715]: Parent detects terminating child

Process[9715]: parent terminating...

The parent program will create a process, which executes the child program using thesystem function execve(3), as contained in child.c. The parent then calls the systemfunction wait to block itself until the OS kernel signals the process to continue again, becauseone of its child processes has just terminated.

8.1.3 Scripts again?

Anything executable, including those shell scripts as discussed in Section 7, can be used tostart a child process in this forking context. We start with the following parent1.c file.

/* parent1.c */

#include <stdio.h>

#include <stdlib.h>

#include <sys/wait.h>

int main(int argc, char * argv[]){

38

if(fork()==0){

char *cmd[] = { "mySpell", argv[1], NULL };

//How many parameters do you see?

if(argc==1)

execve("child", NULL, NULL);

else if(argc==2)

execve("mySpell", cmd, NULL);

else exit(1);

exit(0);

}

printf("Process[%d]: Parent in execution...\n", getpid());

sleep(2);

if(wait(NULL)> 0)

printf("Process[%d]: Parent detects terminating child \n", getpid());

printf("Process[%d]: parent terminating...\n", getpid());

}

Below is the original child.c file.

% more child.c

#include <stdio.h>

int main(){

printf("Process[%d]: child in execution...\n", getpid());

sleep(2);

printf("Process[%d]: child terminating...\n", getpid());

}

We now have the mySpell script that we saw earlier.

% more mySpell

#!/bin/sh

# An improved spelling checker

file=$1

for word in ‘spell $file‘

do

# grep tries to find the line where $word occurs in $file

line=‘grep -n $word $file‘

#print out a line

echo " "

#print out the following line

echo "Missplled word: $word"

#print out the line where the misspelled word occurs

39

echo "$line"

done

We compile the child.c into an executable child,

% cc child.c -o child

then compile the parent1.c program.

% cc parent1.c

If we call it without giving it any parameter, thus argc=1, the executable of parent1.cexecutes the child program.

% ./a.out


Process[31194]: child in execution...

Process[31194]: child terminating...



On the other hand, if we call it with a parameter, child.c, thus argc=2, it calls mySpellto check the spelling of the words as contained in this file.

% ./a.out child.c


’

Missplled word: getpid

4: printf("Process[%d]: child in execution...\n", getpid());

6: printf("Process[%d]: child terminating...\n", getpid());

Missplled word: printf

4: printf("Process[%d]: child in execution...\n", getpid());

6: printf("Process[%d]: child terminating...\n", getpid());



%

This above situation is pretty similar to what you have to do in the final project as well,as different commands need different number of parameters. Even the same command mightneed different parameters to do different things. For example, ls will list all the stuff in thecurrent directory, while ls fake will list the content of the fake subdirectory.

40

Labwork 8.1.3:

1. Play with the programs as shown in this section. You may change some of the stuff,such as the time that the child process waits, to see if there is any change of theprintout.

2. Find out the general format, and discuss the various usage, of those “strange” functions,particularly, the exec family, fork(), getpid(), and wait().

You may need these stuff when completing the project....

You should be ready to tackle the very profitable project.

8.2 An example: Listing directories

After writing some thing simple, let’s now read something that is a lot more complicated,which is taken from the textbook[2, §8].

What we have discussed is something of a file, i.e., the content of a file. Sometimes, wealso need to know something about a file. For example, a Unix service, ls, which we oftenuse, lists the names of files in a directory, together with other information such as their size,permissions, and so on.

There are two aspects of this issue: Since a directory is just a file, we only need to getaccess to this file and print out its content. But, it is necessary to use a system call toget other information such as the sizes of files. We will show how to write such a program,fsize, that is to find out the size of a file, and when the argument is a directory, it willrecursively finds out the sizes of all the files as contained in such a directory.

As we mentioned in § 2.1, a directory is a file that consists of a list of filenames withsome other information, including the location of those files. Technically, a location of a fileis given as an index into another table called the inode list. An inode for a file tells where allthe information about that file, except its file name is kept. Thus, an entry in a directoryabout a file contains only two pieces: a file name and an index number.

However, the format and precise contents of a directory vary with system. Thus, wehave to specify a standard interface to deal with such a variety by dividing the directorylisting task into two pieces. The outer level defines a structure, Direnet, and three routinesopendir, readdir and closedir to provide system-independent access to the file name andthe associated inode number in a directory entry. We will write fsize with this interface inmind.

Then, we will show how to implement such a structure and routines on the directorystructures of Version 7 and System V Unix.

The advantage for such an approach is that, when given a different implementation, weonly need to revise the implementation of such an interface, rather than changing everything.

41

8.2.1 Given a standard interface...

The Dirent structure contains the name of a file and its associated inode index number.The declaration of this structure and other routines are collected in a file dirent.h as thestandard interface we can work with.

#define NAME_MAX 14 /* longest file name */

typedef struct } /* portable directory structure */

long ino;

char name[NAME_MAX];

} Dirent;

typedef struct { /* a minimal directory structure with no buffer */

int fd; /* file descriptor for a directory */

Dirent d; /* the Dirent for this directory */

} DIR;

DIR *opendir(char *dirname);

Dirent *readdir(DIR *dfd);

void closedir(DIR *dfd);

The system call stat takes a file name and returns all of the information in the inodefor that file, or -1 if there is an error.

char *name;

struct stat stbuf;

int stat(char *, struct stat *);

With the above declaration, the way to call it is certainly the following:

stat(name, &stbuf);

which fills the structure stbuf with the inode information for the file with its name beingname. The structure stat, as declared in <sys/stat.h> typically contains the followinginformation:

struct stat {/*inode information returned by stat */

dev_t st_dev; /* divide of inode */

ino_t st_ino; /* inode number */

short st_mode; /* mode bits */

short st_nlinks; /* number of links to file */

short st_uid; /* owner’s user id */

short st_gid; /* owner’s group id */

dev_t st_rdev; /* for special files */

off_t st_size; /* file size i characters */

42

time_t st_atime; /* time last accessed */

time_t st_mtime; /* time last modified */

time_t st_ctime /* time inode last changed */

};

Now, we have some ideas as where the information for file Properties is kept; and also howdoes the OS know who has what access rights to a file.

The types dev_t and ino_t are defined in <sys/types.h> , which should be includedas will.

The st_mode contains a bunch of flags to further describe the nature of the file in question.The following is just a part of such description, which is also defined in <sys/types.h> .

#define S_IFMT 0160000 /* type of file */

#define S_IFDIR 0040000 /* directory */

#define S_IFCHR 0020000 /* character special */

#define S_IFBLK 0060000 /* block special */

#define S_IFREG 0100000 /* regular */

/* ... */

We are now ready to write the program fsize. If the mode st_mode obtained tells usthat it is a file, then its size st_size can be immediately printed. Otherwise, we have toprocess that directory one file at a time. The process can be recursive since such a directorymight contain subdirectory. Let’s look at it.

#include <stdio.h>

#include <string.h>

#include "syscalls.h"

#include <fcntl.h>

#include <sys/stat.h>

#include "dirent.h"

void fsize(char *);

main(int argc, char *argv[]){

if (argc==1)

fsize(".");

else

while (--argc> 0)

fsize(*++argv);

return 0;

}

The function fsize prints the size of the file. If the file is a directory, it calls a functiondirwalk to process all the files within such a directory.

43

int stat(char *, struct stat *);

void dirwalk(char *, void (*fcn)(char *));

void fsize(char *name){

struct stat stbuf;

if(stat(name, &stbuf)==-1){

fprintf(stderr, "fsize: can’t access %s\n", name);

return;

}

if(stbuf.st_mode & S_IFMT)==S_IFDIR)

/* check the st_mode structure */

dirwalk(name, sfize);

/* go ahead with the guts first */

printf("%8ld %s\n", stbuf.st-size, name);

/* prints out information for this directory file */

}

The function dirwalk applies a function, as given in terms of a pointer, to each file (subdirectory)in a directory. The general flow is clear: it opens the directory, goes through all the fileswithin with the function, then closes the directory and returns. To find out the size of a file,it calls fsize indirectly.

#define MAX_PATH 1024

void dirwalk(char *dir, void (*fcn)(char *)){

char name[MAX_PATH];

Dirent *dp; /* portable interface */

DIR *dfd;

/* calls opendir to fill the fstat structure */

if((dfd=opendir(dir))==NULL){

fprintf(stderr, "fsize: can’t access %s\n", name);

return;

}

while((dp=readdir(dfd))!=NULL){

if(strcmp(dp-> name, ".")==0||strcmp(dp-> name, "..")==0)

continue; /* skip itself and parent, see Section 2.1 */

if(strlen(dir)+strlen(dp-> name)+2> sizeof(name))

/* dir/dp-> name/ contains more than MAX_PATH characters /*

fprintf(stderr, "dirwalk: name %s %s too long\n", dir, dp-> name);

else {

sprintf(name, "%s%s", dir, dp-> name);

44

/* fsize is applied in dirwalk in fsize declaration */

(*fcn)(name);

}

}

/* Done when no more files are left */

fclose(dfd);

}

8.2.2 ... and an implementation of the interface

We are done with the standard interface, and now have to get down to the next level toprovide a minimal implementation for this standard interface.

The following one is for Version 7 and System V Unix, two of the more popular versionsof Unix, which uses the following directory information as declared in <sys/dir.h> .

/* See Chapter 4 notes

#ifndef DIRSIZE

#define DIRSIZ 14

#endif

struct direct {/* directory entry */

ino_t d_ino; /* inode number */

char d_name[DIRSIZ]; /* long name does not have ’\0’ */

}

It is clear that in those two versions, the names are rather short. For some other versions,much longer names are allowed which could lead to a more complicated directory structure.

The type ino_t as occurred in the above direct and stat structures is a typedef thatdescribes the index into the inode list. We regularly implement it as a unsigned short

type, but it may change with a different system.Now, we will check out the three basic routines: opendir, readdir and closedir.

The routine opendir opens the directory as a file, verifies its status as a directory,allocates a directory structure for it, and then records the information.

/* fstat is similar to stat, but applies to a file descriptor

rather than a file name */

int fstat(int fd, struct state *);

DIR *openfir(char *dirname){

int fd;

struct stat stbuf;

/* DIR is part of the standard interface */

DIR *dp;

45

if((fd=open(dirname, O_RDONLY, 0))==-1 ||fstat(fd, &stbuf)==-1

||(stbuf.st_mode & S_IFMT)!=S_IMDIR ||(dp=(DIR*) malloc(sizeof(DIR)))==NULL)

return NULL;

dp-> fd=fd;

return dp;

}

The routine closedir simply closes a directory opened by an invocation of opendir.

void closedir(DIR *dp){

if(dp){

/* for close system call, see Section 6.3 of this lab notes */

close(dp-> fd);

/* for the free function, see the address arithmetic unit in Chapter 5 notes */

free(dp);

}

}

The important(:-)) routine readdir uses the read system call to reach each directoryentry. If a directory slot is not currently in use, the inode number is 0, and this position isskipped. Otherwise, the inode number and the associated name will be placed in a static

structure as called for the standard interface, then return a pointer to such a structure.

#include <sys/dir.h>

Dirent *readdir(DIR *dp){

struct direct dirbuf;

static Dirent d;

while(read(dp-fd, (char *)&dirbuf, sizeof(dirbuf))==sizeof(difbuf)){

if(dirbuf.d_info==0) /* slot not in use */

continue;

d.ino=dirbuf.d_ino;

strcpy(d.name, dirbuf.d_name, DIRSIZ);

d.name[DIRSIZ]=’\0’;

return &d;

}

return NULL

}

8.2.3 A little summary

The function fsize is one of those programs that they are not “system programs” in thesense that they are not exclusively written for the system, but they merely make use of the

46

information provided by the system, through the stat structure in this case. For such pro-grams, all the system information should be included as a head file, such as the sys/stat.h

file, but not embedded into these programs. There are at least two advantages for this ap-proach: one is to share all such information consistently among all the files that have a needto use them; and the other is to remove as much redundancy as possible.

The division of the design in terms of a standard interface and a system dependentimplementation is also an important point since this makes the fsize function specificsystem independent.

Another important point for this example is that it demonstrates how to use C, a high-level programming language, to write programs at a system level.

It also demonstrates a few applications of bitwise operators, space allocations, etc..

References

[1] Anderson, P., Just Enough Unix (Fourth Ed.), McGraw-Hill, Boson, MA, 2003.

[2] Kernighan, B., and Ritche, D., The C Programming Language (Second Ed.), PrenticeHall, Englewood Cliffs, NJ, 1988.

47

a gentler introduction to unix

Documents