what is unix? introduction to unix - virginia...

1

Introduction to UNIX

CS 2204Class meeting 1

*Notes by Doug Bowman and other members of the CS faculty at Virginia Tech. Copyright 2001-2003. (C) Doug Bowman, Virginia Tech, 2001- 2

What is UNIX?A modern computer operating systemOperating system:

“a program that acts as an intermediary between a user of the computer and the computer hardware” Software that manages your computer’s resources (files, programs, disks, network, …)e.g. Windows, MacOS

Modern: features for stability, flexibility, multiple users and programs, configurability, etc.

(C) Doug Bowman, Virginia Tech, 2001- 3

Why UNIX?Used in many scientific and industrial settingsHuge number of free and well-written software programsOpen-source OSInternet servers and services run on UNIXLargely hardware-independentBased on standards


Brief history of UNIXKen Thompson & Dennis Richieoriginally developed the earliest versions of UNIX at Bell Labs for internal use in 1970s

Borrowed best ideas from other OssMeant for programmers and computer expertsMeant to run on “mini computers”


Early UNIX HistoryThompson also rewrote the operating system in high level language of his own design which he called B.The B language lacked many features and Ritchie decided to design a successor to B which he called C.They then rewrote UNIX in the C programming language to aid in portability.


UNIX variantsTwo main threads of development:

Berkeley software distribution (BSD)Unix System Laboratories System V

Sun: SunOS, SolarisGNU: Linux (many flavors)SGI: IrixFreeBSDHewlett-Packard: HP-UXApple: OS X (Darwin)…

2


Layers in the UNIX System

Hardware(cpu, memory, disks, terminals, etc.)

UNIX Operating System(process management, memory management,

the file system, I/O, etc.)

Standard Library(open, close read, write, etc.)

Standard Utility Programs(shell, editors, compilers, etc.)

Users

System Interface calls

Library Interface

User Interface

User Mode

Kernel Mode


UNIX StructureThe kernel is the core of the UNIX system, controlling the system hardware and performing various low-level functions. The other parts of the UNIX system, as well as user programs, call on the kernel to perform services for them.


UNIX StructureThe shell is the command interpreter for the UNIX system. The shell accepts user commands and is responsible for seeing that they are carried out.


UNIX StructureOver two hundred utility programs or toolsare supplied with the UNIX system. These utilities (or commands) support a variety of tasks such as copying files, editing text, performing calculations, and developing software.This course will only attempt to introduce a limited number of these utilities or tools!


Getting started


Logins and passwordsYou must have an “account” on the UNIX machine you’re trying to use“login” is your user name (usually some variant of your real name)Your password will not echo as you typeRemember good password practices

3


The shell promptAfter logging in, you get some information about the system, then a shell promptshell: The program you use to send commands to the UNIX systemPrompts take many forms:

$%username@hostname>hostname # %

Must beat the “Tyranny of the blank screen”(C) Doug Bowman, Virginia Tech, 2001- 14

Entering commandsAnytime you see a prompt, you can enter a command for the shellSome commands are a single word

whodatels

Others use additional informationcat textfilels -l


Command syntaxCommands must be entered exactly. If you make a mistake before entering, delete/backspace to fix it. Be careful!command options argument(s)Options: modify command’s executionArguments: often filenames that tell a command what to operate on


Example commands: ls (list)ls –lls –a –lls –alls –a; ls -lls textfilels folderls textfile1 textfile2ls –al textfile


If you don’t get a normal shell prompt…

A program is probably runningIf you see a special program prompt, try to quit the program (quit, bye, exit)If you see nothing, you can

Stop the program with CTRL-z (program will wait until started again)Interrupt the program with CTRL-c (program will usually die)


Ending your sessionAlways log out when you are doneUse the command exit (also sometimes logout or CTRL-d)Note: if you are running a window system, logging out of the shell only ends that shell. You must also log out of the window system using a menu.

1

UNIX filesystem


*Notes by Doug Bowman and other members of the CS faculty at Virginia Tech. Copyright 2001-2003.

(C) Doug Bowman, Virginia Tech, 2001 2

UNIX filesystemThe filesystem is your interface to

physical storage (disks) on your machinestorage on other machinesoutput devicesetc.

Everything in UNIX is a file (programs, text, peripheral devices, terminals, …)There are no drive letters in UNIX! The filesystem provides a logical view of the storage devices


Working directoryThe current directory in which you are workingpwd command: outputs the absolute path (more on this later) of your working directoryUnless you specify another directory, commands will assume you want to operate on the working directory


Home directoryA special place for each user to store personal filesWhen you log in, your working directory will be set to your home directoryYour home directory is represented by the symbol ~ (tilde)The home directory of “user1” is represented by ~user1


UNIX file hierarchyDirectories may contain plain files or other directoriesLeads to a tree structure for the filesystemRoot directory: /No file extensions!

/

tmpusersbin

user1 user2

cs2204joketxt

lab2txtlab1txt


Some Standard Directories/ - root directory/bin – Home of all binaries/dev – device directory/etc – host-specific files and directories/home – users home directories

/home/grads/sgifford/lib – libraries for various languages/sbin – System administration utilities, tools,…/tmp – temporary files/var – Variable data that is changing

2


Path namesSeparate directories by /Absolute path

start at root and follow the treee.g. /users/user1/joketxte.g. ~user1/joketxte.g. ~/joketxt

/

tmpusersbin

user1 user2

cs2204joketxt

lab2txtlab1txt


Path names (cont’d)Relative path

start at working directory.. refers to level above;. refers to working dir.If /users/user1/cs2204is working dir, these both refer to the same file:../joketxt../../user1/joketxt

/

tmpusersbin

user1 user2

cs2204joketxt

lab2txtlab1txt


Changing directoriesChange the working directory with the cd commandcd <directory_path>Use absolute or relative path namescd without a path is equivalent to cd ~

e.g. cd ../../user1/e.g. cd /users/user1/


Output of ls -lFtotal 4lrwxr-xr-x 1 user1 user 18 Aug 28 13:41 home ->

/usr/people/bowman/ -rw-r--r-- 1 user1 user 94 Aug 28 13:42 nothing.txtdrwxr-xr-x 2 user1 user 9 Aug 28 13:40 test_dir/

We’ll keep coming back to this slide!File type

Permissions Owner Group Modify date Filename

Size in bytes


Types of filesPlain (-)

Most filesIncludes binary and text files

Directory (d)A directory is actually a filePoints to another set of files

Link (l): A pointer to another file or directorySpecial: e.g. peripheral devices


Special filesCharacter special files

keyboard, console, etccrw-rw---- 1 lp sys 6, 0 Dec 31 1969 lp0

Block special filescd-rom, disks, etc

brw-rw---- 1 root floppy 2, 0 Dec 31 1969 lp0

3


Creating, moving, renaming, copying, and removing filestouch <file> (create empty file)mv <file1> <file2> (rename)mv <file1> <dir> (move)mv <file1> <dir/file2>

(move & rename)cp <file1>[<file2>|<dir>|<dir/file2>] (copy)rm [-i] <file(s)> (remove)


Creating and removing directoriesmkdir <directory_name>

Create a subdirectory of the current directoryrmdir <directory _name>

Remove a directory (only works for empty directories)

rm –r <directory _name>Remove a directory and all of its contents, including subdirectories


Creating linksln –s <existing_file> <link_name>This command creates a symbolic linkThe file “link_name” will be a pointer to the “existing_file” which may be in another directory or even on another physical machine


File ownershipEach file has a single ownerchown command can be used to change the owner (usually only root user can use this command)There are also various groups to which users can belongGroups may have different permissions than everyone else


File permissionsPermissions used to allow/disallow access to file/directory contentsRead (r), write (w), and execute (x)For owner, group, and world (everyone)chmod <mode> <file(s)>chmod 700 filetxtchmod g+rw filetxt


File modification dateLast time the file was changedUseful information when

There are many copies of a fileMany users are working on a file

touch command can be used to update the modification date to the current date (or to create a file if it doesn’t exist)

4


Looking at file contentscat <filename(s)>

“concatenate”output the contents of the file all at once

more <filename(s)>Output the contents of a file one screen at a timeAllows forward and backward scroll and search


Wildcards in file namesAll of the commands covered here that take file names as arguments can also use wildcards* for any string, e.g. *txt, obj*, a*.*? for any character, e.g. doc?[] around a range of characters, e.g. [a-c]*


Getting help on UNIX commands

These notes only give you the tip of the iceberg for these basic commandsman <command_name> shows you all the documentation for a commandapropos <keyword> shows you all the commands with the keyword in their description

1

Text editing and more basic commands




Text editingSo far, we’ve manipulated files in the file system and viewed their contentsNow we need to know how to modify the contents of filesMost UNIX editors work on plain ASCII text files, not special formatsCommon text editors: vi, emacs, picoWindow systems have own window-based editors


Why vi?Available on practically all UNIX systemsCan be used with a remote login (no windows)Balance of simplicity and power

That’s what your professor had to learn, so, darn it, you’re going to learn it too! ☺


vi basicsInvoke with: vi [filename(s)]The vi editor takes over your screenVarious modes:

command mode – typed characters send commands to systeminsert modeappend mode typing changes the textchange mode


Command modevi starts in this modeFrom other modes, enter command mode by typing EscCommands for:

Cursor movementEditingFile operationsSearchEntering other modes


Cursor movementMove the cursor with j (down), k (up), l (right), h (left), or with arrow keysLarger movements:

4j – move four lines downCTRL-F, CTRL-B – page down, page upw – move to beginning of next word]], [[ - next/previous section0, $ - beginning/end of current line:n – move to line n

2


Editing commandsu – undo last typingx – delete current character*dd – delete current line*dw – delete current word*rx – replace current char. with x*yy – copy current line*p – paste copied/deleted itemsJ – join two lines* can be preceded by a number to do operation multiple times


File operationsZZ, :wq – save and quit:w – save:w filename – save as filename:q – quit:q! – quit without saving:e <filename> - load another file:n – load next file


Search/string – search forward for string?string – search backward for stringn – repeat previous searchN – repeat search opposite direction% - find match of current (, [, {


Entering other modesi – insert text before current char.I – insert text at beginning of linea – append text after current char.A – append text at end of lineO, o – open line above/below currentcw – change (overwrite) current wordC – change text after cursor


Additional infoUNIX in a nutshell, ch. 8 lists all vi commandsTrivia: vi uses all lowercase characters except ‘v’ (and many uppercase and punctuation characters) for commands

Be careful!Use ‘u’, ‘U’, and ‘:q!’ if you mess up

UIAN, ch. 7 covers emacs


More shell commands: communicationtalk – interactive chat with another user (not on lab machines)Mail – text-based email programftp – text-based FTP programtelnet, ssh – connect to other machines directlylynx – text-based web browser

3


More shell commands: processesps – list current processestop – dynamic display of system’s utilization by processeskill – terminate a processtime – keep timing information for a process

1

I/O Redirection & Regular Expressions




Redirecting stdoutInstead of writing to the terminal, you can tell a program to print its output to another file using the > operator>> operator is used to append to a fileExamples:man ls > ls_help.txtEcho $PWD > current_directorycat file1 >> file2


Redirecting stdinInstead of reading from the terminal, you can tell a program to read from another file using the < operatorExamples:Mail [email protected] < messageinteractive_program < command_list


Pipes and filtersPipe: a way to send the output of one command to the input of anotherFilter: a program that takes input and transforms it in some way

wc - gives a count of words/lines/charsgrep - searches for lines with a given stringmoresort - sorts lines alphabetically or numerically


Examples of filteringls -la | morecat file | wcman ksh | grep “history”ls -l | grep “gifford” | wcwho | sort > current_users


What is a regular expression (RE)?

A patternDefines a set of strings of charactersAny string in the set is said to be “matched” by the RE (the RE matches the string)

2


Why REs?Pattern matching is a useful tool in many real-world situations:

Search for a file on a filesystemFind and replace strings in a fileExtract particular data elements from a database

REs are an important part of formal languages - one of the basic CS theory disciplines


UNIX programs that use REsgrep (search within files)egrep (grep but with extended RE’s)vi/emacs (text editors)ex (line editor)sed (stream editor)awk (pattern scanning language)perl (scripting language)


Characters vs. metacharacters

In patterns, characters can be any character except a newlineMetacharacters are special characters that have a special meaningTo use metacharacters as regular characters in a pattern, quote them with the ‘\’ character


Basic vs. Extended RE’sIn basic regular expressions themetacharacters ?, +, {, |, (, and ) have no special meaning (grep).

To make them special, instead use thebackslashed versions \?, \+, \{, \|, $, and $

For extended regular expressions these are interpreted as specialGrep –E egrep


Using egrepegrep pattern filename(s)To be safe, put quotes around your patternExamples:

egrep “abc” file.txt (print lines containing “abc”)egrep -i “abc” file.txt (same, but ignore case)egrep -v “abc” file.txt (print lines notcontaining “abc”)egrep -n “abc” file.txt (include line numbers)


Metacharacters 1Period (.): matches any single character

“a.c” matches abc, adc, a&c, a;c, …“u..x” matches unix, uvax, u3(x, …

Asterisk (*) matches zero or moreoccurrences of the previous RE

Not the same as wildcards in the shell!“ab*c” matches ac, abc, abbc, abbbc, …“.*” matches any string

3


Metacharacters 2Plus (+): matches one or more occurrences of the preceding RE

“ab+c” matches abc, abbc, but not acQuestion mark (?): matches zero or one occurrences of the preceding RE

“ab?c” matches ac, abc but not abbcLogical or (|): matches RE before or RE after bar

“abc|def” matches abc or def


Metacharacters 3Caret (^): means beginning of line

“^D” matches all strings starting with DDollar sign ($) means end of line

“d$” matches all strings ending with dBackslash (\): used to quote othermetacharacters

“file\.txt” matches file.txt but not fileatxt


Metacharacters 4Square brackets ([ ]) indicate a set/range of characters

Any characters in the set will match^ before the set means “not”- between characters indicates a rangeExamples:

“[fF]un” matches fun, Fun“b[aeiou]g” matches bag, beg, big, bog, bug“[A-Z].*” matches any string beginning with a captial letter“[^abc].*” matches any string not starting with a, b, or c


Metacharacters 5Parentheses ( ): used to group REs when using other metacharacters

“a(bc)*” matches a, abc, abcbc, abcbcbc, …“(foot|base)ball” matches football, baseball

Braces ({ }): specify the number of repetitions of an RE

“[a-z]{3}” matches three lowercase letters“m.{2,4}” matches strings starting with m between three and five letters


What do these mean?egrep “^B.*s$” fileegrep “ [0-9]{3} ” fileegrep “num(ber)? [0-9]+” fileegrep “word” file | wc -legrep “[A-Z].*\?” fileWhat about if they were grep?


PracticeConstruct egrep commands that find in file:

Lines beginning with a word of at least 10 charactersLines containing a student ID number in standard 3-part formNumber of lines with 2 consecutive capitalized wordsNumber of lines not ending in an alphabetic characterLines containing a word beginning with a vowel at the end of a sentence

1

UNIX window systems




Why window systems?Increased usability due to

VisibilityGraphical representation of programsSee multiple environments at once

Direct manipulationEnables powerful graphics programs


Window systems and UNIXMost UNIX users can be considered experts, and are fiercely protective of the command line (for good reason)However, all current UNIX systems have a built-in window system - the advantages are inescapable


X WindowsPractically all UNIX window systems are based on X WindowsThe standard version is called X11Very complex system with many partsBasically, X11:

Manages the screen spaceDraws simple graphicsAssigns rectangular regions to various programs


X’s client-server architectureX is actually meant to work over the networkX server: software that runs on the machine where the program’s output will be displayedX client: program running on the same or another machineClient sends drawing and other X commands to the server, which displays the results


Historical use of XUsers sat at “X terminals” - graphical terminals that only knew how to run an X serverThey logged in to other UNIX machines remotely and ran X clients thereThis gave users the benefits of a window system without the need for a full-featured computer on every desk

2


Features of XTransparent remote executionGives programs their own virtual screenIncludes important windowing concepts:

Window damageWindow reveal eventsBacking store

X11 programs are highly portable


Window managersNot part of X11 itselfRun on top of X11Place borders, decorations on windowsHandle input from usersThere are many, many choices with different “look & feel”


Desktop environmentsYet another layer, running on top of window managersComplete the desktop metaphor with:

Iconic access to files and directoriesOverall system menus / toolbarsTrash canetc.

GNOME is one example

1

UNIX shell environments




Shell characteristicsCommand line interface between the user and the systemIs simply a program that automatically starts when you loginWaits for user to type in commands


Main shell featuresInteractivity

aliasesfile-name completion

Scripting languageAllows programming (shell scripting) within the shell environmentUses variables, loops, conditionals, etc.Next lecture


Various UNIX shellssh (Bourne shell, original UNIX shell)ksh (Korn shell)csh (C shell, developed at Berkeley)tcshbash (Bourne again SHell)…Differences mostly in level of interactivity support and scripting details


The Bourne again SHell (bash)We will be using bash and ksh as the standard shells for this classThis will be important for shell scripting assignmentsSuperset of the Bourne shell (sh)Borrows features from sh, csh, tcsh & kshCreated by Free Software Foundation


Changing your shellOn most UNIX machines (and lab):which bash (note path)chsh

On the some machines:which ksh (note path /bin/bash)ypchsh

2


Environment variablesA set of variables the shell uses for certain operationsVariables have a name and a valueCurrent list can be displayed with the envcommandA particular variable’s value can be displayed with echo $<var_name>


Some Environment variablesSome interesting variables: HOME, PATH, PS1, USER, HOSTNAME, PWD

$HOME /home/gradstudents/m/miali$PATH /usr/local/bin:/bin:/usr/bin:/usr/X11R6/bin

$PS1 \u@\h:\w\$$USER sgifford$HOSTNAME avocado.cslab.vt.edu$PWD /home/grads/sgifford


Setting environment variablesSet a variable with <name>=<value>Examples:

TERM=vt100PS1=myprompt>PS1=$USER@$HOSTNAME:PS1=“multiple word prompt> “PATH=$PATH:$HOME/binPATH=$PATH:~

DATE=`date`


AliasesAliases are used as shorthand for frequently-used commandsSyntax: alias <shortcut>=<command>Examples:

alias ll=“ls -lF”alias la=“ls -la”alias m=morealias up=“cd ..”alias prompt=“echo $PS1”


Repeating commandsUse history to list the last 16 commandsUse fc -l <m> <n> to list previously typed commands m through n


Editing on the command lineSome command lines can be very long and complicated - if you make a mistake you don’t want to start all over againYou can interactively edit the command line in several ways if using ksh

set -o vi allows you to use vi commands to edit the command lineset -o vi-tabcomplete also lets you complete commands/filenames by entering a TAB

3


Login scriptsYou don’t want to enter aliases, set environment variables, set up command line editing, etc. each time you log inAll of these things can be done in a script that is run each time the shell is started


Login scripts (continued)For bash, order of files is:/etc/profile ~/.bash_profile~/.bash_login~/.profile

After logout~/.bash_logout


Example .bash_profile (partial)# .bash_profile: executed by bash(1) for login

shellsumask 022

# include .bashrc if it exists

if [ -f ~/.bashrc ]; thensource ~/.bashrc

fi# some ls aliasesalias ll='ls -l'alias la='ls -A'alias l='ls -CF'


Login scripts (continued)For ksh, login shells execute:~/.profile

If ENV is set:That file is executed for each new terminalExample:ENV=$HOME/.kshrc

EXPORT ENV


Background processingAllows you to run your programs in the backgroundsgifford@avocado:~/cs2204/$

emacs vi_practice_edited &sgifford@avocado:~/cs2204/$


stdin, stdout, and stderrEach shell (and in fact all programs) automatically open three “files” when they start up

Standard input (stdin): Usually from the keyboardStandard output (stdout): Usually to the terminalStandard error (stderr): Usually to the terminal

Program command

stdin

stdout

Programs use these three files when reading (e.g. cin, writing (e.g. cout), or reporting errors/diagnostics

1

Basic shell scripting




Shell script/programA series of shell commands placed in an ASCII text fileCommands include

Anything you can type on the command lineShell variablesControl statements (if, while, for)


ResourcesReview UIAN chapter 4Online

Advanced bash-scripting guide http://www.tldp.org/LDP/abs/html/index.htmlBash Reference Manual http://www.gnu.org/manual/bash-2.05a/bashref.htmlksh Reference Manualhttp://www.bolthole.com/solaris/ksh.html


Script executionProvide script as an argument to the shell program (e.g. bash my_script)Or specify which shell to use within the script

First line of script is #!/bin/bashMake the script executable using chmodMake sure the PATH includes the current directoryRun directly from the command line

No compilation is necessary!


Simple example script#!/bin/bash

echo “Hello world!”cd ~pwd Output:

Hello world!/home/grads/sgifford


QuotingQuoting is necessary to use special characters in a variable’s value or string”” - shell only interprets $ and ‘‘ and \

$ - variable substitution` - Command substitution\” - Literal double quote

\ is used to escape charactersecho ‘`date +%D`‘ will print: 10/06/03

’’ - shell doesn’t interpret special charactersecho ‘`date +%D`‘ will print: `date +%D`

2


Shell variables

var refers to the name, $var to the valuet = 100 #Sets var t to value 100echo ”\$t = $t” #will print: $t = 100

Remove a variable with unset varNames begin with alpha characters and include alpha, numeric, or underscore

Numeric

Strings

Arrays

Command line arguments

Functions

Read only


Numeric variablesInteger variables are the only pure numeric variables that can be used in bashDeclaration and setting value: declare -i var=100

Expressions in the style of C:(( expression ))e.g. (( var+=1 ))

+, -, *, /, %, &, |, ~, <, >, <=, >=, ==, !=, &&, ||


String variablesIf you do not use the declare keyword with option –i when using a variable for the first time, it will be a stringvar=100 makes var the string ‘100’.

However, (( var=100 )) will treat var as an integer even though it is a string


Array variablesArray is a list of values

Don’t have to declare sizeReference a value by ${name[index]}

${a[3]}$a (same as ${a[0]})

Use the declare -a command to declare an array

declare –a sportssports=(ball frisbee puck)sports[3]=bat


ArraysArray initializationsports=(football basketball)moresports=(${sports[*]} tennis)

${array[@]} or ${array[*]} refers to the entire array contentsecho ${moresports[*]} produces

football basketball tennis


Command line argumentsIf arguments are passed to a script, they can be referenced by $1, $2, $3, …$0 refers to the name of the script$@ - array filled with arguments excluding $0$# - number of arguments

3


Exporting variablesThe export command, when used with a variable name, allows child processes of the shell to access the variable


OutputWe have already seen echoMore common in other shells including ksh is print (does not exist in bash)echo –n does not print newline after output


Return valuesScripts can return an integer valueUse return NThe variable $? will contain the return value of the last command runCan be used to test conditions


ConditionsIf using integers: (( condition ))If using strings: [[ condition ]]Examples:

(( a == 10 ))(( b >= 3 ))[[ $1 = -n ]][[ ($v != fun) && ($v != games) ]]

Special conditions for file existence, file permissions, ownership, file type, etc.


Conditions (continued…)[[ -e $file]] – File exists?[[ -f $file]] – Regular file?[[ -d $file]] – Directory?[[ -L $file]] – Symbolic link?[[ -r $file]] – File has read permission?[[ -w $file]] – File has write permission?[[ -x $file]] – File has execute perm?[[ -p $file]] – File is a pipe?


If statementsSyntax:if conditionthenstatements

elif conditionthenstatements

elsestatements

fi

optional

4


If statementExampleif [[ -r $fname ]]thenecho ’$fname is readable’

elif [[ -w $fname && -x $fname ]]thenecho ’$fname is writeable and executable’

fi


For loopsSyntax:for var [in list]dostatements

doneIf list is omitted, $@ is assumedOtherwise ${list[*]}where list is an array variable.


For loop examplefor colors in Red Blue Green Yellow Orange Black Gray Whitedoecho $colors

doneecho


While loopsSyntax:while conditiondostatements

doneThe keywords break, continue, and return have the same meaning as in C/C++


Case statementsSyntax:case expression inpattern1)statements ;;pattern2)statements ;;…*)statements ;;

esac


Case statement examplecase $1 in-a)statements related to option a ;;

-b)statements related to option b ;;

*)all other options ;;

esac

5


Command substitutionUse the output of a command in a variable, condition, etc. by:`command`$(command)


ExamplesArguments printed in a for loopReformatting the wc commandPerforming a depth-first search

1

UNIX Development: g++ and make




Overview of development process

Creation of source files (.c, .h, .cpp)Compilation (e.g. *.c *.o) and linkingRunning and testing programs


d.hd.cpp

cl /

c d

.cpp

cl /

c d

.cpp

d.bj

Object files contain machine code but not in executable form.

Object files contain references (calls) to external functions that must be resolved.

Linker programs (Win: LINK, UNIX: ld), are commonly invoked by (C/C++) compilers:

Separate Compilation Steps

–Step 1

source files compiled

to object files

–Step 2

objects files

linked to form

executable

image

file.exe

a.h

a.cpp

a.obj

b.h

b.cpp

b.obj

c.h c.cpp

cl /

c c

.pp

cl /

c c

.pp

c.obj


c.h c.cpp c.obj d.hd.cppfile.exe

cl /

c c

.pp

cl /

c c

.pp

cl /

c d

.cpp

cl /

c d

.cpp

a.h

a.cpp

a.obj

b.h

b.cpp

b.obj

d.bj

A change in one module (requires only 1 recompile:

Changing header files requires recompiling all files dependent upon the header files.


Development tools in UNIXCreation of source files (.c, .h, .cpp)

Text editors (e.g. vi)Revision control systems (e.g. rcs)

Compilation (e.g. *.c *.o) and linkingCompilers (e.g. gcc, g++)Automatic building tools (e.g. make)

Running and testing programsDebuggers (e.g. gdb)

Integrated development environments (IDEs)(C) Doug Bowman, Virginia Tech, 2001 6

Compiling with g++GNU C++ compilerPerforms one or more of the following:

C++ pre-processingCompilationLinking

2


Basic g++ examplesg++ hello.cc (compile hello.cc, produce executable a.out)g++ -o hello hello.cc (compile hello.cc, produce executable hello)g++ -o hello hello.cc other_functions.cc (compile hello.ccand other_functions.cc, produce executable hello)


Using intermediate filesFrom any source file, you can produce an object file to be linked in later to an executable

g++ -c hello.ccg++ -c other_functions.ccg++ -o hello hello.o other_functions.o


Other important g++ options-g: include debugging symbols in the output-l<name>: include a library called libname.a


Include and library pathsThere are default directories in which g++ looks for include files and libraries-I<path>: also look for include files in this directory-L<path>: also look for library files in this directory


Defines in g++Often programs contain conditional parts based on defines:#ifdef DEBUGprintf(“value of var is %d”, var);#endif

You can set preprocessor defines on the command lineg++ -DDEBUG -o prog prog.c


Using make in compilationWith medium to large software projects containing many files, it’s difficult to:

Type commands to compile all the files correctly each timeKeep track of which files have been changedKeep track of files’ dependencies on other files

The make utility automates this process

3


Basic Makefile exampleprogram : main.o iodat.o dorun.og++ -o program main.o iodat.o dorun.o

main.o : main.ccg++ -c main.cc

iodat.o : iodat.ccg++ -c iodat.cc

dorun.o : dorun.ccg++ -c dorun.cc


Basic operation of makeReads a file called [Mm]akefile, which contains rules for building a “target”If the target depends on a file, then that file is builtIf that file depends on a third file, then the third file is built, and so on…Works backward through the chain of dependenciesTargets only built if they are older than the files they depend on


Types of lines in MakefilesDependency or rules linesCommandsMacro assignmentsComments


Dependency/rules linesSpecify a target and a list of prerequisites (optional) for that target

target : prereq1 prereq2 prereq3 …


Command linesFollow dependency linesMUST start with a TAB!Any command that can be run in the shell can be placed here

target : prereq1 prereq2command1command2Special variables in commands:

$@ represents the target$? represents prereqs that are newer than target


Basic Makefile exampleprogram : main.o iodat.o dorun.og++ -o $@ main.o iodat.o dorun.o

main.o : main.ccg++ -c $?

iodat.o : iodat.ccg++ -c $?

dorun.o : $?g++ -c dorun.cc

4


Macro (variable) assignmentsYou can use macros to represent other text in a Makefile

Saves typingAllows you to easily change the action of the Makefile

Assignment:MACRONAME = macro value

Usage: ${MACRONAME}


Simplifying the example Makefile with macros

OBJS = main.o iodat.o dorun.oCC = /usr/bin/g++program : ${OBJS}${CC} -o $@ ${OBJS}

main.o : main.cc${CC} -c $?

iodat.o : iodat.cc${CC} -c $?

dorun.o : dorun.cc${CC} -c $?


Comments and other Makefile notes

Comments begin with a ‘#’Can be placed at the beginning of a line or after a non-comment line

Lines that are too long can be continued on the next line by placing a ‘\’ at the end of the first line


Invoking makeBe sure that your description file:

is called makefile or Makefileis in the directory with the source files

make (builds the first target in the file)make target(s) (builds target(s))Important options:-n: don’t run the commands, just list them-f file: use file instead of [Mm]akefile


Simplifying the example Makefile with macros

OBJS = main.o iodat.o dorun.oCC = /usr/bin/g++program : ${OBJS}${CC} -o $@ ${OBJS}

main.o : main.cc${CC} -c $?

iodat.o : iodat.cc${CC} -c $?

dorun.o : dorun.cc${CC} -c $?


Suffix rulesIt’s still tedious to specifically tell makehow to build each .o file from a .c/.cc fileSuffix rules can be used to generalize such situationsA default suffix rule turns .c/.cc files into .o files by running the command:

${CC} ${CFLAGS} -c $<$< refers to the prerequisite (file.cc)

5


Simplifying the example Makefile again

OBJS = main.o iodat.o dorun.oCC = /usr/bin/g++

program : ${OBJS}${CC} -o $@ ${OBJS}


Other useful Makefile tipsInclude a way to clean up your mess

No dependencies!clean:/bin/rm -f *.o coreInclude a target to build multiple programs

all:program1 program2 program3

1

Development: Revision control




Overview of software development process

Creation of source files (.c, .h, .cpp)RCS, this lecture

Compilation (e.g. *.c *.o) and linkingg++ and make, last lecture

Running and testing programsgdb debugger, next lecture


What is revision control?A way to ensure that edits on files are

Logged / archivedConsistent

Especially applicable when projects include

Multiple filesMultiple editors (developers)Maintenance (corrective and enhancing)


Why do you need revision control?

Software development projects are normally done in teamsMultiple people may be responsible for code in a single fileYou may want to freeze a working version and create a new revision branchYou may want to roll back development to a previous point in time


Overview of RCSMaintains complete revision information for a set of files (not limited to source code)Uses major and minor revision numbers (e.g. 1.1, 2.3)Supports locking files so that only one user can edit at a timeSupports merging two edits of the same fileCan automatically place revision information within the file itself


Key RCS commandsrcs (administer project)ci (check in file)co (check out file)rlog (view the log)rcsdiff (see changes since last revision)For more, see UIAN ch. 19

2


Basic RCS usageCreate a subdirectory RCS of the directory containing the files in questionCheck in the files to create an initial revision 1.1Check out and lock files to edit themCheck files back in to create a new revision and allow others to edit them


Checking out filesAssume we start with file main.cchecked in (RCS/main.c,v)co main.c

Creates read-only file main.c in pwdco -l main.c

Creates writable file main.c in pwdLocks file so others can’t check it out


Checking in filesAssume we’ve locked and edited main.cci main.c

Creates a new revision (e.g. 1.2)Removes main.c from pwd

ci -u main.cCreates a new revisionLeaves read-only main.c in pwd

ci -l main.cCreates a new revision (checkpoint)Allows you to keep editing


Setting the revision numberPerhaps we’ve done major revisions to revision 1.4 of main.c and we want to start a new branch (2) on the revision treeci -r2 main.c


Using keyword substitutionEach time you check in a new revision, RCS keeps information about the author, date/time, a log message (which you provide), etc.You can show this information in files by placing special tags within the files when setting up RCS


Keyword example/** Last revision: $Revision$ by $Author$ on $Date$

* $Log$*/

main(){

...}

/** Last revision: $Revision:

1.1 $ by $Author: bowman $ on $Date: 2001/10/10 20:29:27 $

* $Log: main.c,v $* Revision 1.1 2001/10/10

20:29:27 bowman* Initial revision**/

main(){

...}

3


RCS hintsUsually use ci -u file to keep a read-only copy of file for compilation, etc.Use co -l -rR file to retrieve revision number R of file instead of most recent revision

1

Development: Debugging

CS 2204Class meeting 9-10

*Notes by Mir Farooq Ali and other members of the CS faculty at Virginia Tech. Copyright 2001-2003.

2

Example 1TYPE: Accidental

for (i=0; i<numrows; i++)for (j=0; j<numcols; j++);

pixels++;

3

Example 2TYPE: Missing or improper initialization

int minval(int *A, int n) {int currmin;

for (int i=0; i<n; i++)if (A[i] < currmin)

currmin = A[i];return currmin;

}

4

Example 3TYPE: Dyslexic

int minval(int *A, int n) {int currmin = MAXINT;

for (int i=0; i<n; i++)if (A[i] > currmin)

currmin = A[i];return currmin;

}

5

Example 4TYPE: copy and paste bug

switch (i) {case 1:do_something(1); break;

case 2:do_something(2); break;



default:break;

}6

Example 5TYPE: Accidental

if (foo = 5)foo == 7;

2

7

Example 6TYPE: Abused global

int i = 5;int j;int foo(int j) {

for (i=0; i<j; i++) do_nothing();return j; }

void ineedj(void) {cout << "j is " << j << "\n"; }

main() {int j;j = foo(i);ineedj();

}8

Example 7TYPE: Macro bug

// random returns a random (positive) integer.// Random returns a random integer in the range 0// to n-1.

#define Random(n) random()%nval = Random(j-i+1);

9

Example 8TYPE: model error

char* string1 = "Hello";char* string2 = "World";

if (string1 == string2)do_something();

10

Example 9TYPE: Memory Error

int i;char string[5] = "hello";int j;

11

Program memory

Operating System(kernel)

Use

r Spa

ce

Process 1

Process 2

Process n-1

Process n

Globals/Static Variables

Stack

Heap

Program Text(User Code + Libraries)

12

Example 10TYPE: Memory error

char* ptr;

cin >> ptr;

3

13

Example 11TYPE: Off-by-one error

int i;int array[5];int j;

for (i=0; i<=5; i++)cin >> array[i];

14

Example 12TYPE: Special case error

// Delete the node following the one that ptr is// pointing at.void del_link(lnode* ptr) {

ptr->next = ptr->next->next;}

15

Example 13TYPE: Stack frame error

char *initialize() {char string[80];char* ptr = string;return ptr;

}

main() {char *myval = initialize();do_something_with(myval);

}

16

Example 14TYPE: Stack frame error

char* assign() {return "hello world!";

}

main() {char *ptr = assign();

}

17

Example 15TYPE: Static Vs dynamic data

main() {Record city;lnode *list = NULL;while (data_to_read()) {Readin_data(&city);insert(&city, &list);

}}void insert(Record*& city, lnode*& list) {

lnode* ptr = new lnode;ptr->next = list;list = ptr;prt->data = city;

}

1

Development: Debugging




Overview of software development process

Creation of source files (.c, .h, .cpp)RCS, last lecture

Compilation (e.g. *.c *.o) and linkingg++ and make, a previous lecture

Running and testing programsgdb debugger, this lecture


Why use a debugger?No one writes perfect code first time, every timeDesk checking code can be tedious and error-pronePutting print statements in the code requires re-compilation and a guess as to the source of the problemDebuggers are powerful and flexible


Common debugger functionsRun programStop program at breakpointsExecute one line at a timeDisplay values of variablesShow sequence of function callsCatch signals


The GNU debugger (gdb)Free, like all GNU softwareCommand line debuggerMost common ways to invoke:gdb executablegdb executable coregdb executable process_id


Execution commandslist or l (list code)

listlist mainlist 56

run or r (run program from beginning)runrun file.txt file2.txt

next or n (execute next line, stepping over function calls)step or s (execute next line, stepping into function calls)

2


Breakpoint commandsbreak or b (set a breakpoint)break mainbreak 10

delete or d (delete a breakpoint)deletedelete 2

continue or c (continue execution when stopped)


Program information commandsprint or p (print value)print xprint x*yprint function(x)

display (continuously display value)undisplay (remove displayed value)where (show current function stack)


Miscellaneous commandsset (change a value)set n=3

help or h (display help text)helphelp stephelp breakpoints

quit or q (quit gdb)

1

System programming: file management




Parts of the UNIX OS

Shell

Kernel

Tools & Apps

H/W


UNIX system programmingProgramming that uses special features of the UNIX system (the Kernel)Programs make system calls via librariesTypes of system calls

File I/OProcess managementInter-process communication (IPC)Signal handling


Basic file I/ORemember everything in UNIX is a fileProcesses keep a list of open filesFiles can be opened for reading, writing

Note: Some of the system calls used to interact with the UNIX kernel are also available in other OS’. However, they may be (probably are) implemented much differently and some are not available at all.


Basic file I/OEach file is referenced by a file descriptor(integer)Three files are opened automatically

FD 0: standard inputFD 1: standard outputFD 2: standard error

When a new files is opened, it is assigned the lowest available FDman 2 <systemcall> (for more information)


File I/O system calls - open()fd = open(path, flags, mode)path: char*, absolute or relative pathbasic flags:

O_RDONLY - open for readingO_WRONLY - open for writingO_RDWR - open for reading and writingO_CREAT - create the file if it doesn’t existO_TRUNC - truncate the file if it exists (effectively delete existing file)O_APPEND - only write at the end of the file

2


File I/O system calls - open()fd = open(“myfile”,O_CREAT, 00644)

some other flags:O_NDELAY - open immediatelyO_EXCL – create file if it does not existO_SYNC – delay the opening of the file until there is an attempt to write to file.

mode: specify permissions if using O_CREAT


File I/O system calls - close()return_val = close(fd)

Close an open file descriptorReturns 0 on success, -1 on error


File I/O system calls - read()bytes_read = read(fd, buffer, count)

Read up to count bytes from file and place into bufferfd: file descriptorbuffer: pointer to arraycount: number of bytes to readReturns number of bytes read or -1 if error


File I/O system calls -write()

bytes_written = write(fd, buffer, count)Write count bytes from buffer to a filefd: file descriptorbuffer: pointer to arraycount: number of bytes to writeReturns number of bytes written or -1 if error


File I/O system calls -lseek()

retval = lseek(fd, offset, whence)Move file pointer to new locationfd: file descriptoroffset: number of byteswhence:

SEEK_SET - offset from beginning of fileSEEK_CUR - offset from current offset locationSEEK_END - offset from end of file

Returns offset from beginning of file or -1


Simple file I/O example

3


File I/O using FILEsMost UNIX programs use higher-level I/O functions

fopen()fclose()fread()fwrite()fseek()

These use the FILE datatype instead of file descriptorsNeed to include stdio.h


File I/O using FILEsfopen/fclose – simular to open/close

fopen returns a FILE* mode is ‘r’, ‘w’, etc.man fopen

The function fread/fwrite reads/write nmembelements of data, each size bytes long, from the stream pointed to by stream, storing them at the location given by ptr.

man fread


Simple file I/O example using FILEs


Using datatypes with file I/OAll the functions we’ve seen so far use raw bytes for file I/O, but program data is usually stored in meaningful datatypes (int, char, float, etc.)fprintf(), fputs(), fputc() - used to write data to a filefscanf(), fgets(), fgetc() - used to read data from a fileman <function> (for more info)


Detour: printf/scanfOutput to screen: printfSyntax

printf (control string , argument list )Function Prototype:

int printf( const char, *format, ... ) ;


Control String Formatterschar conversion

%d %i decimal integer%u unsigned decimal integer%o unsigned octal integer%x %X unsigned hexadecimal integer%c character%s string or character array%f float%e %E double (scientific notation)%g %G float or double format%% outputs a '%' character

4


Learning by Exampleprintf("The sum of %d, %d, and %d is %d\n", 65, 87, 33, 65+87+33);

The sum of 65, 87, and 33 is 185printf("Character code %c has ASCII code %d.\n", 'A', 'A');

Character code A has ASCII code 65.printf("Error %s occurred at line %d \n",emsg, lno);

Error invalid variable occurred at line 27Note: emsg and Ino are variables!


Learning by Exampleprintf("Octal form of %d is %o \n", 59, 59);

Octal form of 59 is 73printf("Hexadecimal form of %d is %x \n", 59, 59);

Hexadecimal form of 59 is 3Bprintf("Square root of 2 is %f \n", sqrt(2.0));

Square root of 2 is 1.414214


Learning by Exampleprintf("Square root of 157 is %e \n",sqrt(157.0));

Square root of 157 is 1.252996e+01printf("You scored %d out of %d for %d%%.\n",17, 25, 68);

You scored 17 out of 25 for 68%.


Input: Using scanfSyntax

scanf ( control string , argument list )Function Prototype: int scanf( const char, *format, ... ) ;

SemanticsReturns number of variables assigned values or EOF if an error.Argument list expressions are pointer expressions converted according to the corresponding formatter in the control string.Standard input file stdin (keyboard) supplies the input stream.


Learning by Examplescanf(“%c%c%s”, &c1, &c2, s);

/* Input echo */printf(“\“%c%c %s\””, c1, c1, s);Input Data...into that good night

Output

“.. .into that good night”


Final Thoughts:printf() & scanf() have options other than those covered herein.knowledge of printf() & scanf() is required to understand legacy C code, but they should be avoided when possible, however, we are using them for practice!fprintf and fscanf are the file versions of the same calls: see website for slides

5


Simple file I/O example using FILEs and meaningful

datatypes

1

System programming: process management and IPC




UNIX system programmingProgramming that uses special features of the UNIX systemPrograms make system callsTypes of system calls

File I/OProcess managementInter-process communication (IPC)Signal handling


Processes in UNIXProcess: basic unit of execution

Executing instance of a programHas a process ID (PID)Is in a hierarchy of processes (parents, children)Has its own state/context/memory

Shell commands dealing with processes: ps, top, kill, nice, …


Process managementSystem calls dealing with:

Process creationSetting the program a process executesWaiting for a process to terminateProcess terminationSending signals to a process


Process creationpid = fork()Creates a new child process that is an exact copy of the current process

Same program running at same locationSame variable valuesSame open files

Only difference: child has new PIDReturns 0 in the child processReturns child’s PID in the parent process


Setting the program a process executesexec family of functionse.g. execlp(executable_name, arg0, arg1, …);Replaces the current process with a new process image running the executable specified with the arguments listedNew process retains old PID and any open filesOther functions: execl, execle, execv, execvp, execve

2


Waiting for a process to terminatepid = wait(&status)

Suspends execution of the calling process until any child process terminatesReturns PID of terminating childPuts exit status (int) of child in status

pid = waitpid(pid, &status, options)

Suspends execution of the calling process until a specific child terminates


Using fork/exec/wait togetherpid = fork();if(pid == 0)execl(“./program”, “program”,

arg1, NULL);elsepid = wait(&status);

continue execution


Process terminationexit(status)Terminates the calling processCloses all open file descriptorsReturns status to the parent process


Sending signals to a processretval = kill(pid, signal)Some common signals a user program might send:

SIGINT: interrupt (CTRL-c)SIGKILL: killSIGUSR1, SIGUSR2: user-defined


Inter-process communication (IPC)

Information passing between processesTwo basic paradigms

Message passing: processes send information back and forth in messages/packetsShared Memory: processes share a chunk of physical memory and read/write data there to share that information


IPC with pipesExample of message passingint fds[2]; retval = pipe(fds);Creates two file descriptors (a pipe is a file), the first for reading, and the second for writingHow does another process connect to this pipe?

3


Basic pipe exampleint fds[2]; char s[100];

retval = pipe(fds);pid = fork();if(pid == 0){read(fds[0], s, 100);printf(“Read %s\n”, s);

}

elsewrite(fds[1],

“hello”, 6);

NOTES:data is written/read in order (FIFO)reads block until there’s something to readwrites block if the pipe is fullclosing the writing fd causes EOF to be read on the other end

1

sed, awk, & perl


*Notes by Mir Farooq Ali and other members of the CS faculty at Virginia Tech. Copyright 2003.

© Mir Farooq Ali, 2003 2

sed

Stream editorOriginally derived from “ed line editor”Used primarily for non interactive operations

operates on data streams, hence its name

Usage:sed options ‘address action’ file(s)

Example: sed ‘1$s/^bold/BOLD/g’ foo


sed: Line Addressing

using line numbers (like 1,3p)sed ‘3,4p’ foo.txt

“For each line, if that line is the third through fourth line, print the line”

sed ‘4q’ foo.txt“For each line, if that line is the fourth line, stop”

sed –n `3,4p’ foo.txtSince sed prints each line anyway, if we only want lines 3 & 4 (instead of all lines with lines 3 & 4 duplicated) we use the -n


sed: Line addressing (... continued)

sed –n ‘$p’ foo.txt“For each line, if that line is the last line, print”$ represent the last line

Reversing line criteria (!)sed –n ‘3,$!p’ foo.txt

“For each line, if that line is the third through last line, do not print it, else print”


sed: Context Addressing

Use patterns/regular expressions rather than explicitly specifying line numberssed –n ‘/^From: /p’ $HOME/mbox

retrieve all the sender lines from the mailbox file“For each line, if that line starts with ‘From’, print it.” Note that the / / mark the beginning and end of the pattern to matchls –l | sed –n ‘/^.....w/p’“For each line, if the sixth character is a W, print”


sed: Substitution

Strongest feature of sedSyntax is [address]s/expression1/string2/flag

sed ‘s/|/:/’ data.txtsubstitute the character ‘|’ with the character ‘:’

sed ‘s/|/:/g’ data.txt

global

substitute

2


sed: Using files

Tedious to type in commands at the prompt, especially if commands are repetitiveCan put commands in a file and sed can use themsed –f cmds.sed data.txt

file with commands


awk

Powerful pattern scanning and processing languageNames after its creators Aho, Weinberger and Kernighan (Don’t you love how commands are named?)Most commands operate on entire line

awk operates on fields within each line

Usage:awk options [scriptfile] file(s)

Example: awk –f awk.script foo.txt


awk: Processing model

BEGIN { command executed before any input is read}

{Main input loop for each line of input}END {commands executed after all input is

read}


awk: First example

# Begin ProcessingBEGIN {print "Print Totals"}

# Body Processing{total = $1 + $2 + $3}{print $1 " + " $2 " + " $3 " = "total}

# End Processing

END {print "End Totals"}


Input and output files

Input22 78 44 66 31 70

52 30 44 88 31 66

OutputPrint Totals22 + 78 + 44 = 144

66 + 31 + 70 = 16752 + 30 + 44 = 12688 + 31 + 66 = 185End Totals


awk: command line processing

Input1 clothing 3141

1 computers 9161

1 textbooks 21312

2 clothing 3252 2 computers 12321

2 supplies 2242

2 textbooks 15462

Output1 computers 9161

2 computers 12321

awk ‘if ($2 == "computers“) {print}' sales.dat

3


awk: Other features

Formatted printing using printfConditional statements (if-else)Loops

forwhiledo-while


awk: Associative arrays

Normal arrays use integers for their indicesAssociative arrays with strings as their indicesExample: Age[“Robert”] = 56


awk: Example# salesDeptLoop.awk scriptBEGIN {OFS = "\t"}{deptSales [$2] += $3}END {for (item in deptSales)

{print item, ":", deptSales[item]totalSales += deptSales[item]

} # forprint "Total Sales", ":", totalSales

} # END


Input and output

Input1 clothing 3141

1 computers 9161

1 textbooks 21312

2 clothing 3252

2 computers 12321

2 supplies 2242

2 textbooks 15462

Outputcomputers : 21482

supplies : 2242

textbooks : 36774

clothing : 6393

Total Sales : 66891


awk: Example# salesDeptLoop.awk scriptBEGIN {OFS = "\t"}{deptSales [$2] += $3}END {for (item in deptSales)

{print item, ":", deptSales[item]totalSales += deptSales[item]

} # forprint "Total Sales", ":", totalSales

} # END


Perl

"Practical Extraction and Reporting Language"written by Larry Wall and first released in 1987rumour: name came first, then the acronym"Perl is a language for easily manipulating text, files and processes": originally aimed at systems administrators and developers

4


Features

enables quick development of programsno need to define variable typesportableextensible (module import/export mechanism)powerful "regular expression" capabilitiessimple I/O modelmany modulessupport for static scopingbuilt-in debugger


Common usestext-stream filters

transforming, stripping, annotating, combining

simple text manipulationCommon Gateway Interface (CGI) scriptsreport generationsystem scriptinggeneral solution prototypingHello, World!

print ("Hello, world!\n");print "Hello, world!\n";print STDOUT "Hello, world!\n";


Executing Perl scripts

"bang path" convention for scripts:can invoke Perl at the command line, oradd #!/public/bin/perl at the beginning of the scriptexact value of path depends upon your platform (use "which perl" to find the path)

From the command line:% perlprint "Hello, World!\n";CTRL-DHello, World!


Basics

kinds of variable:scalars, lists, "hashes" (also called "associative arrays" or "dictionaries")some rudimentary support for object-orientation, but not really designed as an OOP languageadvanced perl supports pointers, user-defined structures, subroutine references


Basics (contd)An example:

#!/public/bin/perl

$fruit{"apples"} = 5;$fruit{"oranges"} = 3;$fruit{"lemons"} = 2;$fruit{"limes"} = 2;

@keys = keys(%fruit);

foreach $f (@keys) {print "We have $fruit{$f} $f\n";

}


Control structuresSimilar to that in C:

if () { }if () { } else { }if () { } elsif () { } else { } (note spelling)while () { }do { } while()for (;;) { }

foreach: iterates over each element in a listNo "switch" statement:

must use sequence like "if-elsif-elsif-else"conditional expressions as in C:

non-zero value: truezero value: false

5


using shell commands in Perlexample:

$file_01 = “/home/foobar/ex1.txt”;$file_02 = “/home/foobar/ex2.txt”;…$result = system (“diff $file_01 $file_02”);if ($result == 0) {

# files were the same} else {

# files were different}if we are interested in only the result value and not the output from the command, redirect output to /dev/nullexample: …

system(“diff $file_01 $file_02 >/dev/null”)

1

Basic UNIX system administration




System administrationThus far, we’ve only discussed:

the use of UNIX from an end user point of viewSystem programming - accesses the core OS but doesn’t change the way it operates

System administration is another level: changing the way the system is set up and operates for end usersStrongly related to security issues


The superuserMost sys. admin. tasks can only be done by the superuser (also called the root user)Superuser

has access to all files/directories on the systemcan override permissionsowner of most system files

Shell command: su <username>Set current user to superuser or another user with proper password access


Administration through filesAs you would expect, system settings are stored in filesMost of these are stored in /etcWe’ll look at files related to:

Users and groupsFile systemsSystem initializationSystem upkeep

In section 5 of the man pages


/etc/passwdInformation about system users

bowman:x:65:20:D.Bowman:/home/bowman:/bin/ksh

login name[encrypted password]

user IDgroup ID

“real” name

command interpreterhome directory


/etc/groupInformation about system groups

faculty:x:23:bowman,ribbens,mcquain

group name[encrypted group password]

group ID

list of group members

2


/etc/fstabInformation about file systems

/dev/cdrom /cdrom iso9660 defaults,ro,user,noauto 0 0

mount pointfile system type

file system (local device or remote dir.)

mount optionsother flags


/etc/inittab and /etc/init.d/inittab: configuration file for the init process

Defines “run levels”init.d: Directory containing system initialization scripts

Script rc <n> is run at run level nStarts and stops various services


Configuring the systemOften by directly editing files with a text editorIn some cases there are programs that modify the files for youMany systems also have nice graphical user interfaces that let you manipulate these files indirectly


File System Implementation

A possible file system layout


Internal View of a File SystemBoot Block:

The first block in a UNIX file system, contains the boot program and other initialization information or unused.

Super BlockAlways the second block, contains the complete “catalog” of specific information about the file system, including lists of free memory


Internal View of a File SystemInode list blocks

List of inodes for the file system, contiguous and always follows the super block. The number ofinodes is specified by the system administrator

File access and type information, collectively known as the mode. File ownership information. Time stamps for last modification, last access and last mode modification. Link count. File size in bytes. Addresses of physical blocks.

3


Internal View of a File SystemData blocks

OS files, user data and program files etc.


File system Commandsmount

mount a file systemumount

unmount a file systemfsck

check and repair a Linux file systemsync

flush filesystem buffers


crontabUseful to have a script or command executed without human intervention

a script to verify that the networks are working correctly

cron daemon reads cron configuration files called “crontab” short for “cron table”

parse crontabsfind the soonest command to be rungo to sleep until the command’s execution time has arrived


What’s cron and crontabs?Under UNIX, periodic execution is handled by the cron daemon

read one or more configuration files containing as following

command linestimes at which they are to be invoked(on some systems) login names under which they are to run

crontabs/etc/crontab


Format of Crontab filesSeven or six fields

minute hour day month weekday [username] command

an asterisk matches all possible values, a single integer matches that exact value, a list of integers separated by commas (no spaces) used to match any one of the values two integers separated by a dash (a range)used to match any value within the range


Example crontab

SHELL=/bin/bashPATH=/sbin:/bin:/usr/sbin:/usr/binMAILTO=rootHOME=/

01 * * * * root nice -n 19 run-parts /etc/cron.hourly02 4 * * * root nice -n 19 run-parts /etc/cron.daily22 4 * * 0 root nice -n 19 run-parts /etc/cron.weekly42 4 1 * * root nice -n 19 run-parts /etc/cron.monthly

1

UNIX Review


(C) S. S. Gifford, Virginia Tech, 2003- 2

Study HintsYou should have:

Read all required materialFinished all labsCompleted all assignmentsAttended all lecturesReviewed all notes


Study Hints cont.When writing the final, I

Focused on the details that support major topicsDoes not mean there are no detailsDoes mean that I didn’t try to major on the minors

Did not focus on all commands & optionsDid not focus on C syntaxTried to cover things not in the projectsA few questions assume you attended class!A few questions are from the labsA few questions are from the readings


Study Hints cont.3questions, 5.?? points each, total=~190Multiple choice, true/false, and other innovative scantron questions:

5.) What is the first digit of the number between 14 and 16?6.) What is the second digit of the number between 14 and 16?7.) How many of the following choices are correct? (Enter a number between 0-9.)


No Extra Credit on Exam2 hour test, most will complete in <1h(You do have the extra credit assignment you can do, remember?)


What should you know?What is UNIX vs. LinuxWho is Ken Thompson and Dennis Richie?All commands that were covered outside of just the readings

ls, chmod, egrep, awk, sed, perl, mv, rm, cd, etc etc etc.

2


What should you know cont.Absolute vs. relative paths

Directory traversal

LinksFamiliar with standard directoriesWhat each element in ls means (I-nodes?)File ownership, chmodFile editing (vi, modes, major commands)


What you should know cont.I/O Redirection

> >> | <Pipe and Filter

Regular ExpressionsI’m not going to review this hereSpecial characters, metacharacters

egrep vs grepBe able to do regular expressions


What you should know cont.X Windows, the layers, what each do

Client vs. ServerSee the lab for practical application

Different ShellsAlias, major system variables

Eg, how to change the prompt

Login scripts&

Shell Scripting: At a higher level, no details


What you should know cont.makefiles, gcc

What is a dependancyWhat is a targetFor gcc, what do -H -l –L –o –c mean?

gdb, rcs


What you should know cont.Parts of the UNIX OSFile descriptorsProcess management

fork, exec, wait, waitpid

Inter-process communication (IPC)pipes

Signal handling


What you should know cont.System Administration

Know the files, what they doEg, be able to read a crontab file and say what its doing and when

Be able to read an sed and/or awkscript and tell what it doesWhat is perl?

3


After the test…Exam

Sat, July 3, 1 pm - 3:00 pmGrades should be posted by 5:00pm

May give a curve on final, probably won’tOffice hours immediately after test for questionsGrades finalized by 5:00Will post final letter grades

Assume the 10 point scale,X8.0 to X9.9999 is a plusX0.0 to X1.99999 is a minusNo rounding!

what is unix? introduction to unix - virginia...

Documents