unix overview

Unix Overview

CISC2200, Fall 09

1

Using Unix/Linux System Apply for an account

User name and password Log on and off

through PuTTy, or other telnet/ssh client Linux server: storm.cis.fordham.edu

After log in, you are in the home directory associated with each account

2

Your first encounter: shell

Graphical user interface vs. command line interface

Shell: interactive command interpreter On starts up, it displays a prompt character, and

waits for user to type in a command line On input of a command line, shell extracts

command name and arguments, searches for the program, and runs it.

When program finishes, shell reads next command line….

3

Linux commands Command name and arguments:

Some arguments are file names: cp src dest Some arguments are flags/options: head -20 file Note that “head 20 file” will print initial 10 lines of

file “20”, and file “file” Wild cards: *, ?, []

rm *.o: remove all .o files ?: match any one character [abc]: a or b or c

4

Check/Change Login Shell

Many variations: shell, csh, bash, tcsh, ksh To check the shell you are using

echo $shell echo $SHELL echo $0

login shell: default shell for a user, specified in /etc/passwd

To change login shell chsh <your_user_name>

5

Some useful tips Bash stores the commands history

Use UP/DOWN arrow to browse them Use “history” to show past commands

Repeat a previous command “!<command_no>” or “!<any prefix of previous

command> (the most recent match) Search for a command

Type Ctrl-r, and then a string Bash will search previous commands for a match

File name autocompletion: “tab” key

Shell: how does it work Shell: interactive command interpreter Start a shell session within another one

Just enter command “bash” Use ctrl-d or type exit to terminate a session

How does it find the program ? Environment variable PATH stores a list of paths to

search for programs: “set | grep PATH” or “echo $PATH”, “set” to show all variable settings

Builtin commands: history, set, echo, etc.

Customize your shell environment Modify your shell's startup file (in home dir)

sh, ksh: .profile bash: .profile, .bashrc, .bash_login .bash_profile csh: .cshrc, .login tcsh: .tcshrc, .login Note that these all start with dot

Set environment variables Values of environment variables

In sh, ksh, bash: PATH=$HOME/bin:$PATH PS1="You rang? " export PATH PS1 can also do export PS1="Yes? “

In csh, tcsh: setenv PATH $HOME/bin:$PATH set prompt="You rang? "

Create customized command shorthand Aliases

In sh, ksh, bash: alias ls='ls –F’ alias rm=‘rm –I’: so that you have to confirm the

removal In csh, tcsh

alias ls 'ls –F’

File Systems

11

• File: a sequence of 0 or more bytes containing arbitrary information– Directories are stored as file

Hierarchical file system

12

/ (root)

home

staff

bin

zhang

etc

passwd

dev

cdrom tty24

lib

Home directory & Pathname Absolute pathname, path, specify location

of a file or a directory in the complete file structure /home/staff/zhang is pathname for my home

directory To make life easier:

Working directory (or current directory) concept To check your current directory: pwd

To change your current directory: cd <path name of target directory>

Relative pathname: path names specified relative to current directory

13

“..”: refers to parent dir“.”: current directory“/”: root and seperator in file names“~”: home directory

Getting around in the file system To list files/directories:

ls To create a subdirectory:

mkdir <path name of directory> To remove a directory:

rmdir <path name of directory>

14

File manipulating commands

mv: move a file or directory, or rename a file/directory mv src_path dest_path

cp: copy file or directory cp –r src_dir dest_dir

rm: remove a file or a directory rm <filename> rm –r <dir_name>: remove recursively everything

under the directory

A close look at ls Simply type “ls” will list names of files under

current directory[zhang@storm Demo]$ ls

CCodes README SampleCodes ShellScriptes By default, files are listed in alphabetic order Files with names starting with “.” is not listed

ls <pathname> If <pathname> is a directory name, list files under

the directory

Change ls behavior using flags To list “hidden” files

[zhang@storm Demo]$ ls -a

. .. CCodes .HiddenFile README SampleCodes ShellScriptes

To list files in the order of modification time (most recent first)[zhang@storm Demo]$ ls -t

README ShellScriptes CCodes SampleCodes

Long listing

To get more information about each file[zhang@storm Demo]$ ls -altotal 32drwxr-xr-x 5 zhang staff 4096 2008-01-16 16:01 .drwxr-xr-x 41 zhang staff 4096 2008-01-16 16:01 ..drwxr-xr-x 2 zhang staff 4096 2008-01-16 15:55 CCodes-rw-r--r-- 1 zhang staff 38 2008-01-16 16:01 .HiddenFile-rw-r--r-- 1 zhang staff 53 2008-01-16 15:57 READMEdrwxr-xr-x 2 zhang staff 4096 2008-01-16 15:55 SampleCodes

drwxr-xr-x 4 zhang staff 4096 2008-01-16 15:56 ShellScriptes

Total disc space taken in blocks (1024 Byte)

d means directory

Who has permission to read/write the file User name of the owner and its group

File permissions

Each file is associated with permission info. Differentiate three type of users: owner user, user

from same group as owner, others Three type of access

Read (r): use “cat” to open a file to read, use “ls” to list files/directories under a directory

Write (w): modify the contents of the file Execute (x): run the file, or “cd” to the directory

Trying to snoop into other’s directory[zhang@storm ~]$ ls ../roche/ls: cannot open directory ../roche/: Permission

denied

What’s in a file ?

So far, we learnt that files are organized in a hierarchical directory structure Each file has a name, resides under a directory, is

associated with some admin info (permission, owner)

Contents of file: Text (ASCII) file (such as your C/C++ source code) Executable file (commands) A link to other files, …

To check the type of file: “file <filename>”

Display a text file cat: concatenate input files more, less: display a file in screen by screen

Go forward using PgDn, Return key less: can go forward or backward

head, tail: display the first/last 10 lines of a file head -20 <filename>: display first 20 lines

Some useful file related utilities

Counting # of lines, words and characters in files wc

To search files for lines that match a pattern grep “global warming” articles grep “traditional medicine” articles -v option: lines that don’t match the pattern

Where did I define/access a variable named gNumOfOperations ? grep gNumOfOperations *.[ch]

Sort command

Sort the input into alphabetical order line by line

Many options to control sorting order -r: reverse the normal order -n: sort in numeric order -nr: sort in reverse numeric order +n: sort starting at n+1-th field

Compare file contents Suppose you carefully maintain diff. versions

of your projects (so that you can undo some changes), and want to check what’s the difference. cmp file1 file2: finds the first place where two files

differ (in terms of line and character) diff file1 file2: reports all lines that are different

Standard Input/Output

25

• For each program, three special files are automatically created/opened• By default, all three are set to the terminals• In C++, cin, cout, cerr• In C, extern FILE *stderr, *stdin, *stdout;

Standard input/output/error

0 1

2

Simple example

A very simple C program#include <stdio.h>main() { char yourName[256];

printf ("Your name ?\n"); if (fgets (yourName,256,stdin)==NULL) fprintf (stderr,"No input"); else printf("hello, %s\n", yourName); }

Examples Many Linux prog. reads input from keyboard

and writes output to the screen Command “sort”: read lines from terminal (until

Ctrl-D), sorts them and writes to the screen Very flexible when combined with redirection

and pipes

28

Redirect input/output/error Redirect output to a file:

cat tmpfile1 tmpfile2 > newfile cat tmpfile1 > newfile cat tmpfile2 >> newfile: append output to the file

given Redirect error output:

cat tmpfile 2>error_out.txt Redirect input: cat < tmpfile1 > newfile Note: syntax is different under different shells

29

More on redirection To capture both output and error to same file:

./a.out < tt > dd 2> dd : does not work. Error output is not captured.

./a.out < tt > dd 2>&1 ./a.out < tt 2>dd >&2

To discard output, redirect it to /dev/null /dev/null: a special virtual file, “a black hole” ./a.out > /dev/null 2>&1

Combining commands together How many files are there under current

directory ?ls > tmpwc –l < tmprm tmp

Sort current online user by alphabetic order Is some user login to the system now ? (using

grep)

Pipe: getting rid of temporary file Pipe: connect the output of one program to

the input of another program Any prog. that reads from standard input can

read from pipe, similarly for the standard output who am i | ./a.out | wc knows nothing about redirection and pipe

Rule of composition

Pipe: one of the fundamental contributions of UNIX system

Design programs to be connected with other programs Read/write simple, textual, stream-oriented

formats Read from standard input and write to standard

output Filter: program that takes a simple text

stream on input and process it into another simple text stream on output

Command Pipeline: how ?

Pipe an inter-process communication mechanism

provided by kernel Has a reading end and a writing end Any data write to writing end can be read back

from the reading end Read/write pipe is no different from

read/write files

Reading endWriting end

The Power of Pipe Who is using the most CPU ?

ps -eo pcpu,pid,user,args | sort -k 1 -r | head -10

Command Pipeline: how ?*

Shell set things up create a pipe, “start” two programs

simultaneously, with their input/output redirected to the reading/ending end of pipe

Process related commands

37

The workings of shell* For each command line, shell creates new

child process to run the command Sequential commands: e.g. date; who

Two commands are run in sequence Pipelined commands: e.g. ls –l | wc

Two programs are load/execute simultaneously Shell waits for the completion, and then

display prompt to get next command …

Run program in background To start some time-consuming job, and go on

to do something else wc ch * > wc.out & Shell starts a process to run the command, and

does not wait for its completion (i.e., reads and parses next command)

Shell builtin command: wait Kill a process: kill <processid>

ps command To report a snapshot of current processes:

ps By default: report processes belonging to

current user and associated with same terminal as invoker.

Example:[zhang@storm ~]$ ps PID TTY TIME CMD15002 pts/2 00:00:00 bash15535 pts/2 00:00:00 ps

List all processes: ps -e

BSD style output of psLearn more about the command, using man ps

[zhang@storm ~]$ ps axuUSER PID %CPU %MEM VSZ RSS TTY STAT START TIME COMMAND

root 1 0.0 0.0 2112 672 ? Ss Jan17 0:11 init [3]

root 2 0.0 0.0 0 0 ? S< Jan17 0:00 [kthreadd]

root 3 0.0 0.0 0 0 ? S< Jan17 0:00 [migration/0]

root 4 0.0 0.0 0 0 ? S< Jan17 0:00 [ksoftirqd/0]

root 5 0.0 0.0 0 0 ? S< Jan17 0:00 [watchdog/0]

root 6 0.0 0.0 0 0 ? S< Jan17 0:00 [migration/1]

root 7 0.0 0.0 0 0 ? S< Jan17 0:00 [ksoftirqd/1]

root 8 0.0 0.0 0 0 ? S< Jan17 0:00 [watchdog/1]

root 9 0.0 0.0 0 0 ? S< Jan17 0:00 [migration/2]

Some useful commands

To let process keep running even after you log off (no hangup) Nohup <command> & Output will be saved in nohup.out

To run your program with low priority nice <command> &

To start program at specified time (e.g. midnight) at 2am < file_containing_programs

Other useful commands

43

Getting help To check online manual for a command or a

library call man ls, or man fopen Use PgUp,PgDn, Up Arrow, Down Arrow, Return to

move around GNU’s official documentation format: TexInfo

Use “info ls” for additional description about “ls”

Misc. Commands

Send a file to the printer: lpr <fileName> The file should be of format that the printer

recognizes, e.g., text file, postscript file (.ps)! who: who are logged in the system ?

who –a, or who am i which: show the full path of a command

which bash