linuxos 8th sem
TRANSCRIPT
-
7/29/2019 LinuxOS 8th Sem
1/332
Adapted
Mohammad Ariful HyderAssociate Professor, Dept. of CSE
Dhaka City College, Dhaka.
-
7/29/2019 LinuxOS 8th Sem
2/332
Unix first version created in Bell Labs-1969
Write Unix system in C language-1973 by DenisRitchie
AT&T licenses source code for low cost,Trademarkes Unix name, licensees must createnew name for their operating systems
Many Unix Flavors:, IBM-AIX, Solaris,UnixWare,HP-UX
Unix System Administration & Programming30 January 2013 2
Unix/Linux History
-
7/29/2019 LinuxOS 8th Sem
3/332
Unix Principles
Multiuser, Multitasking
Configuration data stored in text
Ability to chain programs together to performcomplex tasks
Working with Wide variety of computers
High Level of Security
Unix System Administration & Programming30 January 2013 3
-
7/29/2019 LinuxOS 8th Sem
4/332
GNU Project / FSF
GNU (Gnu Not Unix) Project started in 1984 (R.Stallman)
Goal: create free Unix clone
By 1990, nearly all required users applications created(gcc, emacs,..)
Free Software Foundation Non-profit organization that manages GNU project
Unix System Administration & Programming30 January 2013 4
-
7/29/2019 LinuxOS 8th Sem
5/332
GPL- GNU General Public License
Primary license for Open Source Software
Encourages free software
All enhancements and changes to GPL softwaremust also be GPL
Often called copyleft
all rights reversed
Unix System Administration & Programming30 January 2013 5
-
7/29/2019 LinuxOS 8th Sem
6/332
GPL- GNU General Public License-2
freedom to run the program, for any purpose. freedom to modify the program to suit your
needs.
freedom to redistribute copies, either gratis orfor a fee.
freedom to distribute modified versions of theprogram, so that the community can benefitfrom your improvements
Unix System Administration & Programming30 January 2013 6
-
7/29/2019 LinuxOS 8th Sem
7/332
Linux Origins
Linus Torvalds Finnish college student in 1991 create Linux kernel Code was made available over the internet for free
Coders world wide took part in developing the OS
When Linux Kernel combined with GNUapplications, complete free Unix-like OS appears
(GNU/Linux OS)
Unix System Administration & Programming30 January 2013 7
-
7/29/2019 LinuxOS 8th Sem
8/332
Why Linux
Linux and GNU software are distributed under theterms of the GNU Public License Agreement (GPL)
Linux is continually being developed byindividuals throughout the world
Skilled programmers can submit additions andimprovements to the Linux OS to LinuxDevelopment Team over the internet
Unix System Administration & Programming30 January 2013 8
-
7/29/2019 LinuxOS 8th Sem
9/332
Why Linux
Applications are developed that are both free andcommercial
Linux is for Intel based platforms, Power PC, Alpha-based Linux is for Intel based, multiprocessorcompatibility (SMPs)
Fully supported
Many distribution (redhat,suse,fedora,ubuntu ...)
Unix System Administration & Programming30 January 2013 9
-
7/29/2019 LinuxOS 8th Sem
10/332
Red Hat Enterprise Linux
Custom version of recent Linux kernel
Utility and applications
Installation and configuration software
Support available
Unix System Administration & Programming30 January 2013 10
-
7/29/2019 LinuxOS 8th Sem
11/332
Installing Red Hat Linux
Partitioning Hard disks
Primary Partition + Swap partition
Installation method
Network Configuration
IP,Netmask, Gateway, Hostname +Domain name+Name Server
Configuring Boot manager (LILO, Grub) Select Package
Unix System Administration & Programming30 January 2013 11
-
7/29/2019 LinuxOS 8th Sem
12/332
Hard disk partitioning
hda: Primary Master IDE hdb: Primary Slave IDE
hdc: Secondary Master IDE hdd: Secondary Slave IDE
sda: First hard on SCSI bus.
sdb: Second hard on SCSI bus.
Unix System Administration & Programming30 January 2013 12
-
7/29/2019 LinuxOS 8th Sem
13/332
Hard disk partitioning-2
(Primary Partitions): 1 2 3 4
(Logical Partitions): 5 6 7
hdxy IDE sdxy SCSI
x (name of hard disk)
y (partition number )
Example: hda2,hdb5,sda6
Unix System Administration & Programming30 January 2013 13
-
7/29/2019 LinuxOS 8th Sem
14/332
Hard disk partitioning-3
Head
Sector
Cylinders
Track
Sectors
Unix System Administration & Programming30 January 2013 14
-
7/29/2019 LinuxOS 8th Sem
15/332
Hard disk partitioning-4
Series of Sectors addressed by (CHS) First sector name MBR: Master Boot Record
contain: Boot program
Partition Table:4 Primary partitions
1 2 3 4
MBR Partition 1 Partition 3Partition 2 Partition 4
Unix System Administration & Programming30 January 2013 15
-
7/29/2019 LinuxOS 8th Sem
16/332
Hard disk partitioning-5
1 2 3 4
Primary
Partition 1
A 2 FAT32 2000
2001 FAT16 3500
3501 EXT2 6000
6001 NTFS 8000
Status Start FS End
0000
1BEh
1FEh
Partition
Table
Boot
Program
Primary
Partition 2
Primary
Partition 3
Primary
Partition 4
Unix System Administration & Programming30 January 2013 16
-
7/29/2019 LinuxOS 8th Sem
17/332
Hard disk partitioning-6
1 2 3 4
MBR
BootSectors
Primary
Partition 2
Primary
Partition 3
Primary
Partition 4
Primary
Partition 1
M
B
R
P. P. 1
Type: FAT32
P. P. 2
Type: Extended
P. P. 3
Type: Ext2
P. P. 4
Type: NTFS
Unix System Administration & Programming30 January 2013 17
-
7/29/2019 LinuxOS 8th Sem
18/332
Hard disk partitioning-7
M
B
R
D: E: F:C:
P1: FAT32 P2: Extended
MB
R
D: E: F:C:
P1: FAT32 P3: Extended
/ Swap
P2: EXT3
< 1024
Unix System Administration & Programming30 January 2013 18
-
7/29/2019 LinuxOS 8th Sem
19/332
Installation Mode
New or Upgrade
from where
CD
HTTP, FTP, NFS, Hard disk
Kickstart
Unix System Administration & Programming30 January 2013 19
-
7/29/2019 LinuxOS 8th Sem
20/332
Local Login
Text mode login at Virtual consoles
Multiple non-GUI logins are possible through the use ofvirtual consoles
There are by default 6 available virtual consoles
Available through Ctrl-Alt-F[1-6]
Graphical login
If X is running, it is available as Ctrl-Alt-F7
Unix System Administration & Programming30 January 2013 20
-
7/29/2019 LinuxOS 8th Sem
21/332
Running Commands
Command have the following syntax:
command [options] [arguments]
Each item is separated by a space Options modify the commands behavior
Arguments are filenames or other informationneeded by the command
Separated commands with semicolon (;)
Example: date, cal
Unix System Administration & Programming30 January 2013 21
-
7/29/2019 LinuxOS 8th Sem
22/332
Changing Your Password
Password should be changed after first login
From the Red Hat menu, choose
Preferances Password
From a terminal, use passwd
Unix System Administration & Programming30 January 2013 22
-
7/29/2019 LinuxOS 8th Sem
23/332
Linux Graphical Environments
XFree86: standard X windows System
GNOME Desktop environment based on the GTK+
toolkit KDE Desktop environment based on the Qt toolkit
Unix System Administration & Programming30 January 2013 23
-
7/29/2019 LinuxOS 8th Sem
24/332
Linux File Hierarchy Concepts
Files and directory are organized into a single-rooted inverted tree structure
Names are case-sensitive, delimited by the /(forward slash) character
Each shell and system process has a currentdirectory
Unix System Administration & Programming30 January 2013 24
-
7/29/2019 LinuxOS 8th Sem
25/332
Linux File Hierarchy Concepts
Directories may
contain plain files orother directories
Leads to a treestructure for the
filesystem Root directory: /
/
tmpusersbin
doug rocket
cs2204joke.txt
lab2.txtlab1.txt
Unix System Administration & Programming30 January 2013 25
-
7/29/2019 LinuxOS 8th Sem
26/332
Changing Directory
pwd displays the absolute path to the current directory
cd changes directories
- To an absolute path: $ cd /home/issa/work- To relative path:$ cd project/docs
- To a directory one level up: $ cd ..
- To your directory home: $ cd
- To your previous working directory: $ cd -
Unix System Administration & Programming30 January 2013 26
-
7/29/2019 LinuxOS 8th Sem
27/332
Listing Directory Contents
List the content of a current directory or a speifieddirectory
Usage:
ls [options] [files or dirs]
Unix System Administration & Programming30 January 2013 27
-
7/29/2019 LinuxOS 8th Sem
28/332
The Home Directory
Every user has a home directory, typically under/home
Contain user-specific configuration files as well asuser data
Represented by the ~ character
Unix System Administration & Programming30 January 2013 28
-
7/29/2019 LinuxOS 8th Sem
29/332
Absolute pathnames
Absolute pathnames begin with a slash (/)
Complete road map to file location
Can be used anytime you wish to specify a file name
Unix System Administration & Programming30 January 2013 29
-
7/29/2019 LinuxOS 8th Sem
30/332
Relative pathnames
relative pathnames do not begin with a slash (/)
Specifies location relative to your current workingdirectory
Can be used as a shorter way to specify a file name
Unix System Administration & Programming30 January 2013 30
-
7/29/2019 LinuxOS 8th Sem
31/332
pathnames
Separate directories by/
Absolute path
start at root and follow the tree
e.g. /users/doug/joke.txt Relative path
start at working directory
.. refers to level above; . refersto working dir.
If /users/doug/cs2204 isworking dir, this refer to thesame file as previous one
../joke.txt
joke.txt
/
tmpusersbin
doug rocket
cs2204
lab2.txtlab1.txt
Unix System Administration & Programming30 January 2013 31
-
7/29/2019 LinuxOS 8th Sem
32/332
File names
File names may be up to 255 characters
All characters are valid, except the /
It may be unwise to use certain special characters in
File names are case-sensitive
Example: MAIL, Mail, mail, and mAil
Again, possible, but may not be wise
Unix System Administration & Programming30 January 2013 32
-
7/29/2019 LinuxOS 8th Sem
33/332
Copying files and Directories
cp copy files and directories
Usage:
cp [options] file destination
More than one files may be copied at a time if thedestination is a directory
cp [options] file1 file2 dest
Unix System Administration & Programming30 January 2013 33
-
7/29/2019 LinuxOS 8th Sem
34/332
Copying Files and Directories:
The Destination If the destination exists and is a directory, the copy
is placed there with the same name
If the destination exist and is a file, the copyoverwrites the destination file
If the destination does not exist, the copy is createdwith that name
Unix System Administration & Programming30 January 2013 34
-
7/29/2019 LinuxOS 8th Sem
35/332
Moving and Renaming
Files and Directories
mv move and/or rename files and directories
Usage:
mv [options] file destination More than one files may be moved at a time if the
destination is a directory
mv [options] file1 file2 dest
Unix System Administration & Programming30 January 2013 35
-
7/29/2019 LinuxOS 8th Sem
36/332
Moving and Renaming Files and
Directories: the Destination
If the destination exists and is a directory, the sourcefiles or directory is moved there with the same name
If the destination exist and is a file, the source file ismoved to that filename, overwriting the file
If the destination does not exist, the source file ordirectory is renamed with that name
Unix System Administration & Programming30 January 2013 36
-
7/29/2019 LinuxOS 8th Sem
37/332
Removing and Creating Files
rm remove files
Usage:
rm [options] filenames
-i interactive
-r recursive
-f force
touch create empty files or update filestamps
Unix System Administration & Programming30 January 2013 37
-
7/29/2019 LinuxOS 8th Sem
38/332
Creating and Removing Directories
mkdir make a directory
rmdir remove an empty directory
rm r recursively remove a directory and allof its content
Unix System Administration & Programming30 January 2013 38
-
7/29/2019 LinuxOS 8th Sem
39/332
Determining File Content
Files can contain many types of data
Check file type before opening to determine
appropriate command or application to use
Syntax
file [options] filename (s)
Unix System Administration & Programming30 January 2013 39
-
7/29/2019 LinuxOS 8th Sem
40/332
Viewing an Entire Text File
Syntax
cat [options] [file ]
Contents of the files are displayed sequentiallywith no break
Files display concatenated
Unix System Administration & Programming30 January 2013 40
-
7/29/2019 LinuxOS 8th Sem
41/332
Viewing Files One Screenful
At a Time Syntax
less [options] [filename]
Display the contents of a text file one screen at atime
less is the pager used by man
Syntaxmore [options] [filename]
Unix System Administration & Programming30 January 2013 41
-
7/29/2019 LinuxOS 8th Sem
42/332
slocate
output is based on a file index database, updatedonly once every day.
Syntax
slocate [pattern]
locate [pattern]
Example
locate profileAll file or directory pathnames matching the
supplied pattern will be displayed
Unix System Administration & Programming30 January 2013 42
-
7/29/2019 LinuxOS 8th Sem
43/332
Getting Help: man Pages
man display pages from reference manual
Manual contains multiple sections
Key string or key word searches
Examplesman ls
man -k copy
man -f profileman 5 passwd
Unix System Administration & Programming30 January 2013 43
-
7/29/2019 LinuxOS 8th Sem
44/332
Other Help Utilities
command --help
The whatis and apropos
info Text based with sections organized by subject
Command is info section_name
Gnome documentation
Unix System Administration & Programming30 January 2013 44
-
7/29/2019 LinuxOS 8th Sem
45/332
bash Introduction
BourneAgain shell
Developed for GNU project
The de facto standard Linux shell
Backward compatible with Bourne shell (sh) the original (standard) UNIX shell.
Bourne shell (sh) original UNIX shell written by
Steven Bourne at AT&T
Unix System Administration & Programming30 January 2013 45
-
7/29/2019 LinuxOS 8th Sem
46/332
bash Heritage
Bourne shell (sh) original UNIX shell written bySteven Bourne at AT&T
C shell (csh) written by bill joy at UC Berkeley Added many features such as command completion,
history and job control
Korn shell (ksh) written by David Korn at AT&T
Unix System Administration & Programming30 January 2013 46
-
7/29/2019 LinuxOS 8th Sem
47/332
bash Heritage-2
Bourne Again Shell (bash)
Implements many of the extra features found in
csh, ksh Command line completion
Command line editing
Command line history
Sophisticated prompt control
Unix System Administration & Programming30 January 2013 47
-
7/29/2019 LinuxOS 8th Sem
48/332
bash Heritage
C shell (csh) written by bill joy at UC Berkeley
Added many features such as command completion,history and job control
Korn shell (ksh) written by David Korn at AT&T Enhanced C shell (tcsh) a community effort
Unix System Administration & Programming30 January 2013 48
-
7/29/2019 LinuxOS 8th Sem
49/332
bash Heritage-2
Bourne Again Shell (bash)
Implements many of the extra features found in
csh, ksh Command line completion
Command line editing
Command line history
Sophisticated prompt control
Unix System Administration & Programming30 January 2013 49
File Globbing
-
7/29/2019 LinuxOS 8th Sem
50/332
File Globbing
Globbing is wildcard expansion:
* matches zero or more characters
? matches any single character
[a-z] matches a range of characters [^a-z] matches all except the range
Example
$ls /usr/bin/b*
$ls /usr/bin/?b*
$ls a[0-9]
$ls [!Aa]*Unix System Administration & Programming30 January 2013 50
-
7/29/2019 LinuxOS 8th Sem
51/332
The Tab key
Type key to complete command line: For the command name, it will complete a command name
For an argument, it will complete a file name
Examaple: slo
slocate
ls myf
Unix System Administration & Programming30 January 2013 51
-
7/29/2019 LinuxOS 8th Sem
52/332
history
bash stores a history of commands youve entered,which can be used to repeat commands
Use historycommand to see a list of remembered
commands: $ history Use bang character !
!x execute last command begin with x
!2 execute command no 2!! Execute Last command
Unix System Administration & Programming30 January 2013 52
History-2
-
7/29/2019 LinuxOS 8th Sem
53/332
History 2
Use the up and down arrow keys to scroll throughprevious commands
Type to search for a command in
command history (reverse-i-search)`:
To recall last argument from previous command:
. Or
Unix System Administration & Programming30 January 2013 53
-
7/29/2019 LinuxOS 8th Sem
54/332
T i l d e
Tilde (~)
Refer to your home directory:
$cat ~/.bashrc
May refer to another users home directory:
$ ls ~issa/.bashrc
Unix System Administration & Programming30 January 2013 54
-
7/29/2019 LinuxOS 8th Sem
55/332
Variable and String
Parameter/Variable: ($) Substitute the value of variable in a command line
$ cat $HOME/.bash_profile
$ echo $PATH
Curly braces: { }
A string is created for every pattern inside the bracesregardless if any file exist
$ rm hello.{c,o}
Unix System Administration & Programming30 January 2013 55
Command and Math
-
7/29/2019 LinuxOS 8th Sem
56/332
Command and Math
Command output: `` or $() substitute output from a command in a command line
$ echo Hostname: `hostname`
$ echo $(date)
Arithmetic: $[ ]
Substitute result of arithmetic expression in acommand line
$ echoArea: $[ $x * $y ]
Unix System Administration & Programming30 January 2013 56
-
7/29/2019 LinuxOS 8th Sem
57/332
Protecting from Expansion:
Backslash Backslash ( \ ) is the escape character and makes
the next character literal
$ echoYour cost: \$5.00
Used as last character on line to continuecommand on next line:
$ echo This long string will be echoed \
back as one long lineThis long string will be echoed back as one long line
Unix System Administration & Programming30 January 2013 57
-
7/29/2019 LinuxOS 8th Sem
58/332
Protecting from Expansion:
Quotes Single quotes () inhibit all expansion
Double quotes () inhibit all expansion except
$ (dollar sign) including $() ,
` (back tick), command substitution
\ (backslash),single character inhibition
! (exclamation point), history substitution
Unix System Administration & Programming30 January 2013 58
d d k
-
7/29/2019 LinuxOS 8th Sem
59/332
Command Editing Tricks
move to beginning of line
move cursor to end of line
f move cursor to beginning of next word b move cursor to beginning of previous or
current word
delete to beginning of line
deleteto end of line
Unix System Administration & Programming30 January 2013 59
d d k
-
7/29/2019 LinuxOS 8th Sem
60/332
Command Editing Tricks
Editing Modes
By default, bash uses emacs-style key bindings forcommand editing
Type set o vito change key bindings tovi-style
Makevi the default by adding the command above to$HOME/.inputrc
Unix System Administration & Programming30 January 2013 60
-
7/29/2019 LinuxOS 8th Sem
61/332
The bash Shell
Variable
A variable is a label that has a value
Variables are resident in memory
Two types: local and environment Local variables are used only by the shell
Environment variables are passed onto other commands
set to display all variables
env display environment variables
Unix System Administration & Programming30 January 2013 61
-
7/29/2019 LinuxOS 8th Sem
62/332
Local Variable
Conventionally all upper-case
Setting variable value
$ CREDIT=300
$ echo $CREDIT
Common Local Variables HISTFILESIZE determine how many command to be saved
in the history file on logout
COLUMN sets the width of the terminal
LINES sets the height of terminals
PS1 sets the prompt
Unix System Administration & Programming30 January 2013 62
-
7/29/2019 LinuxOS 8th Sem
63/332
Aliases
Aliases let you create shortcuts to command
$ Alias dir=ls laF
Use alias to see all set aliases To see alias value: $ alias dir
$ alias dir=ls laF
Unix System Administration & Programming30 January 2013 63
-
7/29/2019 LinuxOS 8th Sem
64/332
Environment variables
Shell variables exist only in current shell instance
Environment variables passed to subshells
Shell variables can be exported into environment
$ CREDIT=300; export CREDIT
Unix System Administration & Programming30 January 2013 64
-
7/29/2019 LinuxOS 8th Sem
65/332
Common Environment variables
HOMEPath to users home directory
LANG Identification of default language programshould use
PWDUsers current working directory EDITORdefault editor program
LESS options to pass to the command less
TERM terminal type
Unix System Administration & Programming30 January 2013 65
-
7/29/2019 LinuxOS 8th Sem
66/332
Environment variables PATH colon separated list of locations where
commands can be found
which command (not variable) showing path of
executable$which ls
Path to command can also be given explicitly
$ /bin/ls /etc$ ./myls /etc
Unix System Administration & Programming30 January 2013 66
-
7/29/2019 LinuxOS 8th Sem
67/332
Shell Startup Script Script of command executed at login
Include:
Configure the shell by setting local variables
Configure other program through environment variables Establish aliases
Run program on startup
Unix System Administration & Programming30 January 2013 67
-
7/29/2019 LinuxOS 8th Sem
68/332
Shell Startup Script
Login shells are first shells started (i.e. when you login)
Shells launched from a login shell typically are not
login shells Login shells and non-login shells run different startup
script
Unix System Administration & Programming30 January 2013 68
-
7/29/2019 LinuxOS 8th Sem
69/332
Startup Script: order of execution
Login shells /etc/profile
/etc/profile.d ~/.bash_profile
~.bashrc/etc/bashrc
Non-Login shells
~.bashrc
/etc/bashrc
. /etc/profile.d
Unix System Administration & Programming30 January 2013 69
-
7/29/2019 LinuxOS 8th Sem
70/332
/etc/profile
System-wide startup script Paresed by all users with Bourne-style shells,
including bash and sh
Usually sets default PATH variable, user limit, andother variables and settings
bash only source (execute) /etc/profile if the shell is alogin shell
Unix System Administration & Programming30 January 2013 70
-
7/29/2019 LinuxOS 8th Sem
71/332
/etc/profile.d
Some application-specific startup script in thisdirectory
Script called by a for-loop in /etc/profile
Script set up variables and run initializationprocedures
Unix System Administration & Programming30 January 2013 71
-
7/29/2019 LinuxOS 8th Sem
72/332
~/.bash_profile and ~.bashrc For user-specific settings Common to place variable setting, aliases
variables and run initialization procedures
Commands that place output to the screen, such asthe date command should go in .bash_profile not.bashrc
Unix System Administration & Programming30 January 2013 72
-
7/29/2019 LinuxOS 8th Sem
73/332
~/.bash_logout
Execute when exiting a login shell
Useful for running programs automatically atlogout
Example uses: Make backup of files
Delete temporary files
Display date and time of logout
Unix System Administration & Programming30 January 2013 73
Overview of vi and vim
-
7/29/2019 LinuxOS 8th Sem
74/332
Overview ofvi and vim
vi:the visual editor, standard Linux and Unix editor
vim: the vi improved standard Red Hat editor
On Red hat operating systems, thevi commandinvokesvim
Derived from earlier Unix editors
ed->ex->vi->vim
Unix System Administration & Programming30 January 2013 74
Starting of vi and vim
-
7/29/2019 LinuxOS 8th Sem
75/332
Starting ofvi and vim
To start vim : vi filename
If the file exists, the file is opened and the content are
displayed If the file doesnt exists, vi create it when the edits are
saved for the first time
Unix System Administration & Programming30 January 2013 75
-
7/29/2019 LinuxOS 8th Sem
76/332
Three Modes ofvi and vim
Command mode Cursor movement
Change, delete, yank, put
Insert mode Type in new text
Return to command mode with
Ex mode
Configuring, exiting, saving
Search and replace
Unix System Administration & Programming30 January 2013 76
Three Modes of vi and vim
-
7/29/2019 LinuxOS 8th Sem
77/332
Three Modes ofvi and vim
Unix System Administration & Programming
command
mode
insert
mode
last line
mode
iESC :
wrong command
30 January 2013 77
-
7/29/2019 LinuxOS 8th Sem
78/332
Cursor Movement
Insert mode h left, j down , k up , l rightwword ahead, b word back
Arrow keys also work
a append after the cursor
i insert before the cursor
Aappend to end of line
I insert at beginning of line o open a line below
O open a line above
Unix System Administration & Programming30 January 2013 78
-
7/29/2019 LinuxOS 8th Sem
79/332
Command mode
takes you from insert mode back tocommand mode
cc change line, dd delete line,yyyank (copy) line cw change word, dw delete word,yw yank word
Use p or P to put (paste) copied or deleted data
Unix System Administration & Programming30 January 2013 79
Undoing Changes
-
7/29/2019 LinuxOS 8th Sem
80/332
Undoing Changes
u undo most recent change
U undo all changes to current line since the cursorlanded on the line
redo last undone change
Unix System Administration & Programming30 January 2013 80
Searching for Text
-
7/29/2019 LinuxOS 8th Sem
81/332
Searching for Text
/text search downwards for text
?text search Upwards for text
n continue search in the same direction
N continue search in the opposite direction
Unix System Administration & Programming30 January 2013 81
S i d E iti d
-
7/29/2019 LinuxOS 8th Sem
82/332
Saving and Exiting: ex mode
:wq save changes
:q or q! abonden changes
:w save without exit :e! abandon changes without exit
:wq! Forcing save changes
:w! Forcingsave without exit
Unix System Administration & Programming30 January 2013 82
A f t i k
-
7/29/2019 LinuxOS 8th Sem
83/332
A few tricks
5dd delete 5 line (a number can precede any of thetow character change, delete, yank, or put command
x delete a character
rc replace a character with c
Rreplace character for character until
Unix System Administration & Programming30 January 2013 83
U
-
7/29/2019 LinuxOS 8th Sem
84/332
Users
Every user of the system is assigned a unique User IDnumber
Users names and uid are stored in /etc/passwd
User are assigned a home directory and a programthat is run when they log in (usually shell)
Users cannot read, write or execute each others files
without permission
Unix System Administration & Programming30 January 2013 84
-
7/29/2019 LinuxOS 8th Sem
85/332
Groups
Users are assigned to groups with unique group IDnumber (gid)
gids are stored in /etc/group
Each user is given their own private group They can also be added to other groups to gain
additional access
all users in a group can share files that belong tothe group
Unix System Administration & Programming30 January 2013 85
-
7/29/2019 LinuxOS 8th Sem
86/332
The root users
The root user: a special administrative account
Sometimes called the superuser
root has complete control over the system
An unlimited capacity to damage the system! You should not log in as root without a very good reason
Normal (unprivileged) users potential to do damage islimited
Unix System Administration & Programming30 January 2013 86
-
7/29/2019 LinuxOS 8th Sem
87/332
User and Group ID Numbers
User names map to user ID numbers
Group names map to group ID numbers
Data stored on the hard disk is stored numericallyAuthentication information is stored in plain text
files:
/etc/passwd
/etc/shadow
/etc/group
Unix System Administration & Programming30 January 2013 87
-
7/29/2019 LinuxOS 8th Sem
88/332
System Users and Groups
Server programs such as web or print serverstypically run as unprivileged users, not as root
Examples: mail, lp, nobody
Running programs in this way limits the amount ofdamage any single program can do to the system
Unix System Administration & Programming30 January 2013 88
-
7/29/2019 LinuxOS 8th Sem
89/332
Changing Your Identity
To change your password, run passwd
Insecure passwords are rejected
To start a new shell as a different user: su
su
su username
su - username
Unix System Administration & Programming30 January 2013 89
-
7/29/2019 LinuxOS 8th Sem
90/332
Users Information Commands
Find out who you are
whoami
Find out what groups you belong to groups
Find out who is logged in
users, who, w
Login / reboot history last
Unix System Administration & Programming30 January 2013 90
-
7/29/2019 LinuxOS 8th Sem
91/332
The Linux Security Model
Users and groups are used to control access to filesand resources
Users log in to the system by supplying their user
name and password Every file on the system is owned by a user and
associated with a group
Every process has an owner and group affiliation ,and can only access the resources its owner orgroup can access
Unix System Administration & Programming30 January 2013 91
-
7/29/2019 LinuxOS 8th Sem
92/332
Linux File Security
Every file and directory has permission set thatdetermine who can access it
Permission are set for:
The owner of the file (called user) The group members
All others
Permissions that are set are called read, write andexecute permissions
Unix System Administration & Programming30 January 2013 92
-
7/29/2019 LinuxOS 8th Sem
93/332
Linux Process Security
When a process accesses a file the user and groupof the process are compared with the user andgroup of the file
If the user matches, the user permissions apply If the group matches, but the user doesnt, the group
permissions apply
If neither match, the other permission apply
Unix System Administration & Programming30 January 2013 93
-
7/29/2019 LinuxOS 8th Sem
94/332
Examining Permissions
File permissions may be viewed using ls$ ls l /bin/login-rwxr-xr-x 1 root root 19080 Apr 1 18:26 /bin/login
File type and file access permission are symbolizedby a 10-character string
Unix System Administration & Programming30 January 2013 94
-
7/29/2019 LinuxOS 8th Sem
95/332
Using Categories
File permissions are established for each of threeuser categories. Each category also has a one-lettersymbol:
u - the files owners (user) g - other users in the files group
o everyone else (others)
Unix System Administration & Programming30 January 2013 95
-
7/29/2019 LinuxOS 8th Sem
96/332
File Permission Types
Four symbols are used when displayingpermission:
r : permission to read a file or list a directorys content
w : permission to write to a file or create and removefiles from a directory
x : permission to execute a program file or change into adirectory and do long listing of the directory
- : no permission (in place of the r, w, or x)
Unix System Administration & Programming30 January 2013 96
-
7/29/2019 LinuxOS 8th Sem
97/332
Examining Directories
The first character in the long listing distinguishdirectories (d) from regular files (-)$ ls l /bin
drwxr-xr-x 2 root root 4096 Apr 1 18:26 /bin/
$ ls l /bin/df
-rwxr-xr-x 1 root root 4096 Apr 1 18:26 /bin/df
Others file types indicators exist
Unix System Administration & Programming30 January 2013 97
-
7/29/2019 LinuxOS 8th Sem
98/332
Examining User Permissions
Characters 2,3, and 4 in the access mode identifyuser permissions$ ls l /bin/login
-rwxr-xr-x 1 root root 19080 Apr 1 18:26 /bin/login
This file can be read, written, and executed by itsowner, and read and executed by everyone else
Unix System Administration & Programming30 January 2013 98
-
7/29/2019 LinuxOS 8th Sem
99/332
Examining group Permissions
Characters 5,6, and 7 in the access mode identifygroup permissions$ ls l README
-rw-rw-r-- 1 issa teachers 9080 Oct 1 8:26 README
This file can be read and written by people ingroup teachers, but it cannot be executed.
Unix System Administration & Programming30 January 2013 99
-
7/29/2019 LinuxOS 8th Sem
100/332
Examining Other Permissions
Characters 8,9, and 10 in the access mode identifypermissions for others:$ ls l README
-rw-rw-r-- 1 issa teachers 9080 Oct 1 8:26 README
This file can be read by people outside groupteachers, but it cannot be changed or executed
Unix System Administration & Programming30 January 2013 100
Changing Permissions
-
7/29/2019 LinuxOS 8th Sem
101/332
Changing Permissions
Symbolic method To change access mode :
$ chmod [-R] mode file
Changes the access mode of files and directories
Unix System Administration & Programming30 January 2013 101
Changing Permissions
-
7/29/2019 LinuxOS 8th Sem
102/332
Changing Permissions
Numeric method Uses a three-digit mode numbers
First digit specifies owners permission
Second digit specifies group permission
Third digit represents others permission Permissions are calculated by adding :
4 (for read)
2 (for write)
1 (for execute)
Example
$ chmod 640 myfile
Unix System Administration & Programming30 January 2013 102
-
7/29/2019 LinuxOS 8th Sem
103/332
File and directory permissions
Read and write for all is the default (666) for filesand 777 for directory
umask Can be used to withhold permission upon
file creation Non-privileged users umask is 0002 and the
roots umask is 0022
File will have permissions of 664
Default Mode 666
umask -002
New File Mode 664
Unix System Administration & Programming30 January 2013 103
-
7/29/2019 LinuxOS 8th Sem
104/332
Special Permissions
Special permissions: a fourth permission set (inaddition to user/group/other),Applicable in fourcases:
SUID (set user ID) for an executable, Processes aregranted access to system resources based on user whoowns the file.
SGID (set group ID) for an executable, Same with SUID
except group is affected. SGID a directory: Files created in that directory will
have their group set to the directory's group.
Unix System Administration & Programming30 January 2013 104
-
7/29/2019 LinuxOS 8th Sem
105/332
Special Permissions
Sticky bit for a directory, If set on a directory, then a usermay only delete files that he owns or for which he hasexplicit write permission granted, even when he has
write access to the directory. (e.g. /tmp )
Unix System Administration & Programming30 January 2013 105
SUID Permissions
-
7/29/2019 LinuxOS 8th Sem
106/332
SUID Permissions
When you login, your login shell process valuesare your user ID and group ID E.g., if you run passwd (owned by root), THE user ID is
your ID, not root; then how can it update /etc/passwdfile owned by root ?
SUID bit enables this functionality
When an executable file with set user ID (SUID)permission is executed, command run with permissionof the owner of the command, not executor of thecommand
File permission of /bin/passwd is r-sr-sr-x
Unix System Administration & Programming30 January 2013 106
Partitions and Filesystems
-
7/29/2019 LinuxOS 8th Sem
107/332
Partitions and Filesystems
Disk drives are divided into partitions
Partitions are formatted with filesystems alloyingusers to store data
Default filesystem: ext3, The Third Extended LinuxFilesystems
Other common filesystems:
ext2 , msdos, FAT, NTFS
Iso9660(typically used for CDs)
Unix System Administration & Programming30 January 2013 107
Inode
-
7/29/2019 LinuxOS 8th Sem
108/332
Inode
An inode tables contains a list of all files in an ext2or ext3 filesystem
An inode (index nodes) is an entry in the table,
containing information about a file (themetadata), including:
File type, permission, link count, UID, GID
The files size and various time stamps
Pointers to the files data blocks on disk
One inode is associated with each file.
The system uses inodes as the definition of a file.
Unix System Administration & Programming30 January 2013 108
Directory
-
7/29/2019 LinuxOS 8th Sem
109/332
Directory
The computers reference for a file is the inodenumber
The human way to reference a file is by file name
A directory is a mapping between the humanname for the file and the computers inodenumber
5 apples
4 oranges
7 home
2 .
Unix System Administration & Programming30 January 2013 109
cp and inode
-
7/29/2019 LinuxOS 8th Sem
110/332
cp and inode
The cp command:
allocates a free inode number, placing a new entry inthe inode table
Create a directory entry, referencing the file human filename to the inode number
Copies data into new file
Unix System Administration & Programming30 January 2013 110
mv and inode
-
7/29/2019 LinuxOS 8th Sem
111/332
mv and inode
If the destination of the mvcommand is on thesame file system as the source, the mvcommand:
Creates a new directory entry, with the news file name
Deletes the old directory entry with the old file name Has no impact on the inode table(except for a time
stamp) or the location of the data on the disk: no data ismoved!
Unix System Administration & Programming30 January 2013 111
rm and inode
-
7/29/2019 LinuxOS 8th Sem
112/332
rm and inode
The rm command:
Decrement the link count, thus freeing the inodenumber to be reused
Places data bocks on the free list Removed the directory entry
Data is not actually removed, but will beoverwritten when the data blocks are used by
another file
Unix System Administration & Programming30 January 2013 112
-
7/29/2019 LinuxOS 8th Sem
113/332
Symbolic (or Soft) links
A symbolic link point to another file
Ls l display the link name and the refenced file
$ ls l pf
lrwxrwxrwx 1 issa issa 11 Oct 1 8:26 ps/etc/passwd
File type: l for symbolic link
Syntax
ln
s filename [linkname]
Unix System Administration & Programming30 January 2013 113
-
7/29/2019 LinuxOS 8th Sem
114/332
Hard links
One physical file on the filesystem
Each link references the files inode
File is present in the filesystem as long as at least
one link remain Cannot span drives or partitions
Syntax
ln filename [linkname]
Unix System Administration & Programming30 January 2013 114
-
7/29/2019 LinuxOS 8th Sem
115/332
The 7 Fundamental filetypes
- regular file
d directory
l symbolic link
b block special file
c character special file
p named pipe
s socket
Unix System Administration & Programming30 January 2013 115
Checking Free Space
-
7/29/2019 LinuxOS 8th Sem
116/332
Checking Free Space
df report disk space usage
Report total kilobytes, kilobytes used, kilobytesfree per file system
-h display sizes in easier to read units
du report disk space usage
Report kilobytes used per directory
Includes subtotals for each subdirectory -s option only report single directory entry
Also takes the -h option
Unix System Administration & Programming30 January 2013 116
-
7/29/2019 LinuxOS 8th Sem
117/332
Find command
List files and pathnames matching a specific filename or criteria
Syntax
find [conditions] [action] Recurses down in file tree in dirs
Optionally executes command on each line ofoutput
Unix System Administration & Programming30 January 2013 117
-
7/29/2019 LinuxOS 8th Sem
118/332
Finding and processing Files
-exec action execute a command
Syntax
find [conditions] -exec cmd {} \;
Found files are passed to cmd as arguments
{} is a place holder for the filename
The escaped semicolon delimit the generatedcommands
Unix System Administration & Programming30 January 2013 118
-
7/29/2019 LinuxOS 8th Sem
119/332
Finding and processing Files
Search Option Description
-name filename Finds files with the given filename
size +n Finds files with the size n-amin +n Finds files that were accessed n minutes
ago
-mtime +n Finds files that were modified n days ago
Unix System Administration & Programming30 January 2013 119
-
7/29/2019 LinuxOS 8th Sem
120/332
Finding Examples
Example: find . -print
find all files in the current directory (and allsubdirectories) & display the results
find . -name *.c -print
find all files in the current directory (and allsubdirectories) with the extension .c & display theresults
Unix System Administration & Programming30 January 2013 120
d l
-
7/29/2019 LinuxOS 8th Sem
121/332
Finding Examples
find / -name myfile -print This command tells the find command to look
inside the / directory and every subdirectory to
look for a file or directory with the name myfileand to display each match it finds
find . -name foo* -amin -10 -print This command tells the find command to look
inside the current directory and every subdirectory
to look for a file with name begin with foo thathave been accessed in the last 10 minutes and todisplay each match it finds
Unix System Administration & Programming30 January 2013 121
d l
-
7/29/2019 LinuxOS 8th Sem
122/332
Finding Examples
Example: (for student)
find . -size +5000k
find . -name "*.tmp" -exec rm {} \;
Unix System Administration & Programming30 January 2013 122
i bl di
-
7/29/2019 LinuxOS 8th Sem
123/332
Mounting Removable Media
Removable media must be mounted before thedata can be accessed
CD often will be mounted automatically
Use the mount command to mount:mount /media/floppy (for mount floppy)
Media must be unmounted before being removed
CDs: eject Floppies: umount /media/floppy
Unix System Administration & Programming30 January 2013 123
F i fl
-
7/29/2019 LinuxOS 8th Sem
124/332
Formatting a floppy
One of the this command depending on whichfilesystem i want to use :
mkfs t ext2 /dev/fd0
mk2fs /dev/fd0mkfs t vfat /dev/fd0
mformat a:
Unix System Administration & Programming30 January 2013 124
Wh A hi Fil
-
7/29/2019 LinuxOS 8th Sem
125/332
Why Archive Files
Archiving places many files into one target file
Easier to back up, store, and transfer
tar -standard Linux archiving command
Syntaxtar cvf archive_name files
c create a new archive
v produce verbose messages
f archive_name is the name of new file
Options do need a leading dash
Unix System Administration & Programming30 January 2013 125
I ti A hi
-
7/29/2019 LinuxOS 8th Sem
126/332
Inspecting Archives
Syntax
tar tf archive_name.tar
tar tvf archive_name.tar
First form display a list of all files in the archive
Thevcause a long listing of each file in the archive
Unix System Administration & Programming30 January 2013 126
E t ti A hi
-
7/29/2019 LinuxOS 8th Sem
127/332
Extracting an Archive
Syntax
tar xvf archive_name.tar
The archive will be extracted in the currentdirectory
Change to the target directory first
Files maintain their hierarchy relative to the
current directory
Unix System Administration & Programming30 January 2013 127
Wh U Fil C i ?
-
7/29/2019 LinuxOS 8th Sem
128/332
Why Use File Compression?
Results in smaller file size
Text files can be compressed up to 75%
Binary files usually dont compress much
tar -archives are often compressed
Unix System Administration & Programming30 January 2013 128
C i Utiliti
-
7/29/2019 LinuxOS 8th Sem
129/332
Compression Utilities
gzip,gunzip
Standard linux compression utility
Up to 75% compression for text files
bzip2,bunzip2
newer linux compression utility
Generally archives better compression than gzip
Unix System Administration & Programming30 January 2013 129
U i C i
-
7/29/2019 LinuxOS 8th Sem
130/332
Using Compression
Sample compression commands
gzip termcap
gzip -v termcap
gunzip -c termcap.gz | wc
l
gunzip termcap
Unix System Administration & Programming30 January 2013 130
C i hi
-
7/29/2019 LinuxOS 8th Sem
131/332
Compressing archives
Often tar archive are compress
tar can compress/uncompress archives
Compression switches- use during creation andextraction
z forgzip compression
j forbzip2 compression
Unix System Administration & Programming30 January 2013 131
TAR ith C i
-
7/29/2019 LinuxOS 8th Sem
132/332
TAR with Compression
file1
sub-
directory
.tar .tar.gz
tar gzip
Unix System Administration & Programming30 January 2013 132
tar to Unformatted Floppies
-
7/29/2019 LinuxOS 8th Sem
133/332
tar to Unformatted Floppies
Floppies can used like tape drives
Low level format required
File system not needed
Use tar to write to the floppy
/dev/fd0 is the destination
Floppy cannot be mounted
Example:$ tar czvf /dev/fd0 mydir
Unix System Administration & Programming30 January 2013 133
Standard Input and Output
-
7/29/2019 LinuxOS 8th Sem
134/332
Standard Input and Output
Linux provides three I/O channel to processes
Standard input (0): keyboard is default Standard output (1): terminal windows is default
Standard error (2) : terminal windows is default
Unix System Administration & Programming30 January 2013 134
Redirecting Input and Output
-
7/29/2019 LinuxOS 8th Sem
135/332
Redirecting Input and Output
standard input, output, and error can bereconnected to alternate locations
Shell redirection operators allow standard I/O channels
to be redirected to/from a file
Pipes allow standard I/O channels to connected to theinput or output of programs
Unix System Administration & Programming30 January 2013 135
Redirecting Output
-
7/29/2019 LinuxOS 8th Sem
136/332
Redirecting Output
In order to study redirecting standard output, anderror , we will use the find command.
find /etc name passwd
This command will search for all files namedpasswd in /etc and its subdirectory
By default both the standard output and standard
error are displayed on the screen.
Unix System Administration & Programming30 January 2013 136
Redirecting Standard Output
-
7/29/2019 LinuxOS 8th Sem
137/332
Redirecting Standard Output
Redirecting standard output with > find /etc name passwd > findresult
standard error is still displayed on the screen.
If the target file of file redirection with > alreadyexists, the existing file will be overwritten
To append data to an existing file, use >> to redirect
instead of>
Unix System Administration & Programming30 January 2013 137
Redirecting Standard Error
-
7/29/2019 LinuxOS 8th Sem
138/332
Redirecting Standard Error
Redirect standard error with 2>
Example : redirect standard error to a file: find /etc name passwd 2> finderrors
Standard outputs still displayed on the screen.
To append data to an existing file, use 2>> toredirect instead of 2>
Unix System Administration & Programming30 January 2013 138
-
7/29/2019 LinuxOS 8th Sem
139/332
Redirecting Both StandardOutput and Error Redirection of standard output and standard error
can be performed simultaneously find /etc name passwd 2> error > result
Each I/O channel can be redirect ed to differentfiles, or the same file:
find /etc name passwd > alloutput 2>&1 Or find /etc name passwd &> alloutput
Unix System Administration & Programming30 January 2013 139
-
7/29/2019 LinuxOS 8th Sem
140/332
Redirecting Input Redirect standard Input with < Some commands only operate on standard input
tr doesnt accept filenames as arguments- it require its
input to be redirected from somewhere
Tr A-Z a-z < .bash_profile
This command will translate the uppercase character inbash_profile to lowercase
Unix System Administration & Programming30 January 2013 140
-
7/29/2019 LinuxOS 8th Sem
141/332
Using pipes To connect processes Pipes (the | character) let you redirect output fromon command to become input to another command
ls /usr/lib | less
The mail command takes the message contents asstandard input
ls l | mail s here the listing issa
Can create pipelines a powerful feature of linux cut f1 -d: passwd | sort -r | less
Unix System Administration & Programming30 January 2013 141
-
7/29/2019 LinuxOS 8th Sem
142/332
tee Let you tee a pipe: redirect output to a file while still
piping it to another program
set | tee set.out | less
Output from set command is written to file set.out whilealso being piped to less
Unix System Administration & Programming30 January 2013 142
-
7/29/2019 LinuxOS 8th Sem
143/332
tee Let writing to output and simultaneously date | tee file1 file2
Thu Jun 10 11:10:34 CEST 2007
cat file1
Thu Jun 10 11:10:34 CEST 2007
cat file2
Thu Jun 10 11:10:34 CEST 2007
uptime | tee -a file2
11:10:51 up 21 days, 21:21, 57 users, load average: 0.04, 0.16, 0.26
cat file2
Thu Jun 10 11:10:34 CEST 2007
11:10:51 up 21 days, 21:21, 57 users, load average: 0.04, 0.16, 0.26
Unix System Administration & Programming30 January 2013 143
-
7/29/2019 LinuxOS 8th Sem
144/332
String Processing Tools
head head command display first few line (default : 10
lines) of text in a file
head /tmp/output.txt Use n or --lines parameter to change number of
line displayed
head n 20 /tmp/output.txt
Unix System Administration & Programming30 January 2013 144
-
7/29/2019 LinuxOS 8th Sem
145/332
String Processing Toolstail tail command display last few line (default : 10
lines) of text in a file
tail /etc/passwd Use n or --lines parameter to change number of
line displayed
tail n 5 /etc/passwd
Use fto follow the end of a text file as it changes tail f 5 make.out
Used to watch log filesUnix System Administration & Programming30 January 2013 145
-
7/29/2019 LinuxOS 8th Sem
146/332
String Processing Tools
wc
word count also counts lines and characters
wc story.txt
39 237 1901 story.txt Use l for only line count
Use w for only word count
Use c for only character count
Unix System Administration & Programming30 January 2013 146
-
7/29/2019 LinuxOS 8th Sem
147/332
String Processing Toolssort Sort text to stout original file unchanged
sort [option] file(s)
Common option:-r Reverse sort to sort descending
-n Numeric sort
-f Ignore case of characters in strings
-u unique (remove duplicate lines in output)
-t x Use x as field separator
-k pos1 sort from field pos1
Unix System Administration & Programming30 January 2013 147
-
7/29/2019 LinuxOS 8th Sem
148/332
String Processing Toolssort$ cat bfishdog
animalBird
$ sort -r b
fishdogbirdanimal
Unix System Administration & Programming30 January 2013 148
-
7/29/2019 LinuxOS 8th Sem
149/332
String Processing Toolssort Sort numeric file as text
$ cat c5412 this line should go last
998 this line should go second50 this line should go first
999 this line should go third
$ sort c50 this line should go first
5412 this line should go last
998 this line should go second
999 this line should go third
Unix System Administration & Programming30 January 2013 149
-
7/29/2019 LinuxOS 8th Sem
150/332
String Processing Toolssort Sort numeric file as numbers
$ cat c5412 this line should go last
998 this line should go second50 this line should go first
999 this line should go third
$ sort -n c50 this line should go first
998 this line should go second
999 this line should go third
5412 this line should go last
Unix System Administration & Programming30 January 2013150
String Processing Tools
-
7/29/2019 LinuxOS 8th Sem
151/332
St g ocess g oo s
uniq Remove successive, duplicate lines in a file
Can use in conjunction with sort to remove all
duplicated (or use sort u) Use c to count number of occurrences of
duplicate data
Unix System Administration & Programming30 January 2013 151
String Processing Tools
-
7/29/2019 LinuxOS 8th Sem
152/332
g g
uniqJustsorted:
$ cat a | sortbird
bird
dog
dog
fishfish
fly
sort | uniq:
$ cat a | sort | uniqbird
dog
fish
fly
Unix System Administration & Programming30 January 2013 152
-
7/29/2019 LinuxOS 8th Sem
153/332
String Processing Tools
cut Display specific column of file data
cut f4 result.dat
Common option: -f Specifies field or column
-d Specifies field delimiter (default is TAB)
cut f3 d: /etc/passwd
-c Cuts by characters
cut c2-5 /usr/share/dict/words
Unix System Administration & Programming30 January 2013 153
-
7/29/2019 LinuxOS 8th Sem
154/332
String Processing Tools
cut
Example output second word on each line: Delimiter: space
Fields: 2
$ cat athe quick brown fox
jumped over a quick brown fox
$ cut -f 2 -d ' ' a
quickover
Unix System Administration & Programming30 January 2013 154
-
7/29/2019 LinuxOS 8th Sem
155/332
String Processing Tools
cut
Example output characters 1-3, 5, 7-end
$ cat a
the quick brown fox
jumped over a quick brown fox
$ cut -c 1-3,5,7- a
theqick brown foxjume over a quick brown fox
Unix System Administration & Programming30 January 2013 155
-
7/29/2019 LinuxOS 8th Sem
156/332
String Processing Tools
tr, paste tr -character translator
tr A-Z a-z < .bash_profile
This command will translate the uppercase character inbash_profile to lowercase
paste - merge line of files
Paste file1 file2
Unix System Administration & Programming30 January 2013 156
-
7/29/2019 LinuxOS 8th Sem
157/332
Version Comparison with
diff Compares two files for differences
diff area.c /tmp/area.c
33c33< x = y + 2;
----
> x = y +4;
33c33 indicated line where files differ < indicates line in first file
> indicates line in second file
Unix System Administration & Programming30 January 2013 157
-
7/29/2019 LinuxOS 8th Sem
158/332
Spell checking with aspell Interactive spell-checker Easy way to check spelling in a file
aspell check letter.txt
Can create personal dictionaries
Unix System Administration & Programming30 January 2013 158
Regular Expressions
-
7/29/2019 LinuxOS 8th Sem
159/332
You can use and even administer Linux systemswithout understanding regular expressions butyou will be doing things the hard way
Regular expressions are endemic to Unix
vi, ed, sed, and emacs
Awk, Tcl, Perl and Python
grep, egrep, fgrep
Unix System Administration & Programming30 January 2013 159
-
7/29/2019 LinuxOS 8th Sem
160/332
Downside of Regular Expressions
-
7/29/2019 LinuxOS 8th Sem
161/332
There is considerable variation from utility toutility
The shell is limited to fairly simple metacharactersubstitution (*,?, []) and doesnt really support regex
Regex in vi are also fairly limited Regex in sedare not exactly the same as regex in Perl, or
Awk, orgrep, or egrep
This puts the onus on the user to examine the man
page or other documentation for these utilities todetermine which flavor of regex are supported
Unix System Administration & Programming30 January 2013 161
So How Do We Build a Regex?
-
7/29/2019 LinuxOS 8th Sem
162/332
The simplest regex is a normal character c , for example, will match a c anywhere while an a will
do the same for an a
The next thing is a . (period)
This will match any single occurrence of any characterexcept a newline
For example . will match a z or an e or a ? or evenanother .
w.n will match win, wan, won, wen, wmn, went, andwanton as well as w*n and w9n
Unix System Administration & Programming30 January 2013 162
Protecting Regex Metacharacters
-
7/29/2019 LinuxOS 8th Sem
163/332
Since many of the special characters used in regexsalso have special meaning to the shell, its a goodidea to get in the habit of single quoting yourregexs
This will protect any special characters from beingoperated on by the shell
If you habitually do it, you wont have to worry aboutwhen it is necessary
Unix System Administration & Programming30 January 2013 163
Multiple Occurrences in a Pattern
-
7/29/2019 LinuxOS 8th Sem
164/332
The * (asterisk or star) is used to define zero ormore occurrences of the single character precedingit
abc*d will match abd, abcd, abccd, abcccd, or even
abcccccccccccccccccccccccccccccccccccd Note the difference between the * in a regex and the
shells usage
In a regex, a * only stands for zero or more occurrences of a
single preceding character, In the shell, the * stands for any number of characters that
may or may not be different
Unix System Administration & Programming30 January 2013 164
Specifying Begin or End of Line
-
7/29/2019 LinuxOS 8th Sem
165/332
The ^ specifies the beginning of a line
^The then will match any The that are the firstcharacters on a line
The $ matches the end of line
well$ will match well only if they are the lastcharacters on a line prior to the NEWLINE character
Note that well (notice the space at the end) wouldNOT match well$
^Ken$ would only match a line that started withKen and then had no other characters on the line
Unix System Administration & Programming30 January 2013 165
Character Classes [ ]
-
7/29/2019 LinuxOS 8th Sem
166/332
The square brackets [ ] are used to define characterclasses
[aA]wk will match awk or Awk
Ranges can also be specified in character classes
[1-9] is the same as [123456789]
[abcde] is equivalent to [a-e]
You can also combine multiple ranges
[abcde123456789] is equivalent to [a-e1-9]
Note that the - character has a special meaning in acharacter class BUT ONLY if it is used within a range, [-123] would match the characters -, 1, 2, or 3
Unix System Administration & Programming30 January 2013 166
Negating a Character Class
-
7/29/2019 LinuxOS 8th Sem
167/332
The ^, when used as the first character in acharacter class definition, serves to negate thedefinition
For example [^aeiou] matches any character except a,e, I, o, or u
Used anywhere else within a character class, the ^simply stands for a ^
[ab^&] would match a a, b, ^, or &
Note also that within a character class, the ^ does notstand for beginning of line
Unix System Administration & Programming30 January 2013 167
Escaping Special Characters
-
7/29/2019 LinuxOS 8th Sem
168/332
Even though we are single quoting our regexs so the
shell wont interpret the special characters,sometimes we still want to use a special characteras itself
To do this, we escape the character with a \(backslash)
Suppose we want to search for the charactersequence 8*9*
Unless we do something special, this will match zero ormore 8s followed by zero or more 9s, not what we want
8\*9\* will fix this - now the asterisks are treated asregular characters
Unix System Administration & Programming30 January 2013 168
Reading a Regex
-
7/29/2019 LinuxOS 8th Sem
169/332
If you get in the habit of literally reading a regex, itwill be much easier for you to determine what onedoes
^Ken could be read as matching the word Ken at the
beginning of a line A better way to read it is the beginning of a line followed
by a capital K followed by an e followed by an n
^corn$ would be read as the beginning of a line
followed immediately by a c followed by an o followed byan r followed by an n followed immediately by aNEWLINE
Unix System Administration & Programming30 January 2013 169
Alternation
-
7/29/2019 LinuxOS 8th Sem
170/332
Regex also provides an alternation character ( | )for matching one or another subexpression
(K|T)en will match Ken or Ten
^(From|Subject): will match the From and Subject lines
of a typical email message It matches a beginning of line followed by either the
characters From or Subject followed by a :
The parenthesis ( ) are used to limit the scope of
the alternation At(ten|nine)tion then matches Attention or
Atninetion, not Atten or ninetion as would happenwithout the parenthesis - Atten|ninetion
Unix System Administration & Programming30 January 2013 170
Optional Items
-
7/29/2019 LinuxOS 8th Sem
171/332
The ? (question mark) specifies an optionalcharacter, the single character that immediatelyprecedes it
For example, if I am looking for the month of July, it maybe specified a July or Jul
I could use (July|Jul) to search or I could use July?
Unix System Administration & Programming30 January 2013 171
Repetition
-
7/29/2019 LinuxOS 8th Sem
172/332
The * (asterisk or star) has already been seen tospecify zero or more occurrences of theimmediately preceding character
+ (plus) means one or more abc+d will match abcd, abccd, or abccccccd but will
not match abd while abc?d will match abd and abcdbut not abccd
Unix System Administration & Programming30 January 2013 172
Regex Summary
-
7/29/2019 LinuxOS 8th Sem
173/332
Character Name Meaning
.[][^]
dotcharacter classnegated character class
any one characterany character listedany character not listed
^$
\
caretdollar
backslash less-thanbackslash greater-than
position at start of lineposition at end of line
position at beginning of wordposition at end of word
?*+
{n,m}
question markasterisk or starplus signn to m
matches optional preceding charactermatches zero or more occurrencesmatches one or more occurrencesmatches m to n occurrences
|()
\1, \2,
bar, orparenthesis
backreference
matches either expression it separateslimits scope of | or enclosessubexpressions for backreferencingMatches text previously matched withinfirst, second, etc set of parenthesis
Unix System Administration & Programming30 January 2013 173
grep
-
7/29/2019 LinuxOS 8th Sem
174/332
grep comes from the ed search command globalregular expression print or g\re\p
This was such a useful command that it waswritten as a standalone utility
There are two other variants, egrep andfgrep thatcomprise thegrep family
grep is the answer to the moments where you
know you want a the file that contains a specificphrase but you cant remember its name
Unix System Administration & Programming30 January 2013 174
grep Family
-
7/29/2019 LinuxOS 8th Sem
175/332
Syntax
grep [-hilnw] [-e expression] [filename]
egrep [-hiln] [-e expression] [-f filename] [expression][filename]
fgrep [-hilnx] [-e string] [-f filename] [string] [ filename]
-h - Do not display filenames
-i - Ignore case
-l - List only filenames containing matching lines
-n - Precede each matching line with its line number -w - Search for the expression as a word (grep only)
-x - Match whole line only (fgrep only)
Unix System Administration & Programming30 January 2013 175
Family Differences
-
7/29/2019 LinuxOS 8th Sem
176/332
grep - uses regular expressions for patternmatching
fgrep - file grep, does not use regular expressions,only matches fixed strings but can get search
strings from a file egrep - exponential grep, uses a more powerful set
of regular expressions
Unix System Administration & Programming30 January 2013 176
Regex in the grep Family
-
7/29/2019 LinuxOS 8th Sem
177/332
The following one-character regexs match a singlecharacter
c - an ordinary character
\c - an escaped special character . * [ \ ^ $ A \ followed by < > ( ) { or }
A . (period)
[string] any single character contained within the
brackets
Unix System Administration & Programming30 January 2013 177
Regexs for grep and egrep
-
7/29/2019 LinuxOS 8th Sem
178/332
Regular
ExpressionMeaning
c\m^$.
[xyz]
[a-z][^]
\nr*r+r?
r1r2
r1|r2\(r\)(r)
Normal (nonmeta) characterEscape a characterStart of LineEnd of LineAny single character except NEWLINEAny of x, y, z,
RangeAny single character not listedThe nth tagged expressionZero or more r'sOne or more r's (egrep only)Zero or one r's (egrep only)Concatenation: r1 followed by r2
r1 or r2 (egrep only)Tagged regular expression r (grep only)Regular expression r
Unix System Administration & Programming30 January 2013 178
grep Family Expressions
-
7/29/2019 LinuxOS 8th Sem
179/332
Regular
Ex ression Matchesgrep, fgrep, and egrep
x
xyz
Ordinary characters match themselves (NEWLINES andmetacharacters excluded)Ordinary strings match themselves
grepand egrep
\m^$.
[xy^$x]
[^xy^$z][a-z][^a-z]
r*r1r2
Matches literal charactermStart of lineEnd of lineAny single characterAny of x, y, ^, $, or z
Any one character other than x, y, ^, $, or zAny single character in given rangeAny single character not in rangezero or more occurrences of regex rMatches r1 followed by r2
Unix System Administration & Programming30 January 2013 179
sed command
-
7/29/2019 LinuxOS 8th Sem
180/332
sed stream editorA stream editor is used to perform basic text
transformations on an input stream (a file orinput from a pipeline).
Replace some substring with another$ cat abird barks
mouse runs
$ sed 's/barks/flies/' < abird flies
mouse runs
Unix System Administration & Programming30 January 2013 180
sed command
-
7/29/2019 LinuxOS 8th Sem
181/332
Replace some characters with others Replacing b with Q, i with X
$ cat abird barks
mouse runs
$ cat a | sed 'y/bi/QX/'QXrd Qarks
mouse runs
Unix System Administration & Programming30 January 2013 181
Advanced example
-
7/29/2019 LinuxOS 8th Sem
182/332
Calculate number of hits per remote host inApache log file, most active hosts first.
Line format:
159.148.123.123 - - [28/Oct/2004:18:11:36 +0300]
"GET /somefolder/file.php HTTP/1.1" 200 127602
"-" "Opera/7.54 (X11; Linux i686; U) [en]"
Unix System Administration & Programming30 January 2013 182
Advanced example
-
7/29/2019 LinuxOS 8th Sem
183/332
$ cat access_log | cut -d ' ' -f 1 | sort | uniq -c | sort n r
First, cut out the host part (1st field), sort it get the number of repeated lines before the line (uniq c :
prefix lines by the number of occurrences), sort it numerically, reversed so that largest number comes
first
Output:
348698 159.148.111.222123485 159.148.48.5412313 80.123.123.4
Unix System Administration & Programming30 January 2013 183
Processes
-
7/29/2019 LinuxOS 8th Sem
184/332
Operating systems are about managing resources Files are about storage (and I/O) resources
Processes are about CPU resources
Definition :A process is a single program running in its own
virtual address space, it receives a share (or time slice) of the CPU
Processes and commands are not the same A simple command (ls) generates a single process
complex commands or scripts can invoke several simultaneousexecuting processes
Unix System Administration & Programming30 January 2013 184
Process types
-
7/29/2019 LinuxOS 8th Sem
185/332
Interactive
initiated and controlled by terminal session
can accept input from user as it runs
can output results to the terminal Linux hasjob control to manage processes
Daemons
Typically started by init process
Unix System Administration & Programming30 January 2013 185
Job control for interactive processes
-
7/29/2019 LinuxOS 8th Sem
186/332
foreground process bydefault interactive processes run in the foreground
and the shell must wait until they complete
onlyone process can be running in the foreground for
each user. When a command is entered, thye shell willnot process further input until the process is completeand the shell prompt is redisplayed
But Remember that Linux is a multi-user system -
foreground and background relate to user sessions.Hence multiple foreground processes can be running.
Unix System Administration & Programming30 January 2013 186
Job control for interactive processes
-
7/29/2019 LinuxOS 8th Sem
187/332
Background process if a process has no output to terminal and will take
some time to run, rather than waiting, it can be run as abackground process
Once started in the background control returnsimmediately to the shell.
a user can initiate multiple simultaneous backgroundprocesses
with the bash shell following a command with an &places in the background
eg makewhatis &
Unix System Administration & Programming30 January 2013 187
Job control for interactive processes
-
7/29/2019 LinuxOS 8th Sem
188/332
The jobs command show what process aresuspended or running in the background
Moving a process from the foreground to thebackground takes two steps A process running in the foreground can be suspended
( ctrl-Z)
A suspended process can be placed in the background(use bg command)
One suspended or background process can bebrought to the foreground (fg command)
Unix System Administration & Programming30 January 2013 188
Job control for interactive processes
-
7/29/2019 LinuxOS 8th Sem
189/332
Command Meaning
regular_command Runs this command in the foreground.command & Run this command in the background
jobs Show commands running in thebackground.
Ctrl+Z Suspend (stop, but not quit) a processrunning in the foreground (suspend).
Ctrl+C Interrupt (terminate and quit) a processrunning in the foreground.
%n Every process running in the backgroundgets a number assigned to it.
bg %n Reactivate a suspended program in thebackground.
fg %n Puts the job back in the foreground.kill End a process
Unix System Administration & Programming30 January 2013 189
Compound Commands
-
7/29/2019 LinuxOS 8th Sem
190/332
List of commands seperated by semi-colons List inside () to run inside a subshell
$ ( cd /usr; du )
Unix System Administration & Programming30 January 2013 190
Daemons
-
7/29/2019 LinuxOS 8th Sem
191/332
server process running in the background often started at boot time
offer service to other processes
examples ftpd - file transfer process daemon
cron daemon - scheduling daemon
httpd - web server
Unix System Administration & Programming30 January 2013 191
Process Attributes
-
7/29/2019 LinuxOS 8th Sem
192/332
the ps and top commands can be used to look atcurrent processes
PID - process ID : each process has a unique ID
PPID parent process ID : The process that start a process
nice value - priority (-20 highest to 19 = lowest)
TTY associated terminal (TTY teletype terminal)
Unix System Administration & Programming30 January 2013 192
Process Attributes
-
7/29/2019 LinuxOS 8th Sem
193/332
Displaying process information ps
PID TTY TIME CMD
4245 pts/7 00:00:00 bash
5314 pts/7 00:00:00 ps
ps aux or ps auxw options most commonly used
ps -ef | grep username
top (process monitoring)
uptime
pstree
Unix System Administration & Programming30 January 2013 193
Process States
-
7/29/2019 LinuxOS 8th Sem
194/332
process can be in one of many states
R runnable (on the run queue)
S sleeping
T - stopped D uninterupptible sleep
Z defunt (zombie) process
Unix System Administration & Programming30 January 2013 194
Process life cycle
-
7/29/2019 LinuxOS 8th Sem
195/332
On startup a single process (called init, with PID 1)is created
All subsequent process are derived from by thefork-exec mechanism When a process forks it creates an exact copy of itself
called the child processes
The exec system call them places the image of the newprocess over the copy before allowing it to be scheduled
to run when a process dies (finishes or is killed) it sends a
signal to its parent
Unix System Administration & Programming30 January 2013 195
Illustration of Process Control Calls
-
7/29/2019 LinuxOS 8th Sem
196/332
Unix System Administration & Programming30 January 2013 196
Process Control A process is a program in execution.
P id tifi d b th i id tifi
-
7/29/2019 LinuxOS 8th Sem
197/332
Processes are identified by their process identifier, an
integer. Process control system calls
fork creates a new process
execveis used after a fork to replace on of the two processessvirtual memory space with a new program
exit terminates a process
A parent maywait for a child process to terminate;waitprovides the process id of a terminated child so that the parentcan tell which child terminated.
wait3 allows the parent to collect performance statistics aboutthe child
Azombie process results when the parent of a defunctchild process exits before the terminated child.
Unix System Administration & Programming30 January 2013 197
Process Control (Cont.)
-
7/29/2019 LinuxOS 8th Sem
198/332
Processes communicate via pipes; queues of bytesbetween two processes that are accessed by a filedescriptor.
All user processes are descendants of one original
process, init. init forks agetty process: initializes terminal line
parameters and passes the users login name tologin. login sets the numeric user identifierof the process to
that of the user
executes a shellwhich forks subprocesses for usercommands.
Unix System Administration & Programming30 January 2013 198
Signals
-
7/29/2019 LinuxOS 8th Sem
199/332
Facility for handling exceptional conditionssimilar to software interrupts
Most desirable way to end a process is to let itend normaly
The interrupt signal, SIGINT, is used to stop acommand before that command completes(usually produced by ^C)
If all else fail, send a kill signal
Unix System Administration & Programming30 January 2013 199
Signals
-
7/29/2019 LinuxOS 8th Sem
200/332
Signal name number Meaning
SIGTERM 15 Terminate the process in anorderly way.
SIGINT 2 Interrupt the process. A processcan ignore this signal.
SIGKILL 9 Interrupt the process. Aprocess can not ignore this signal.
SIGHUP 1 For daemons: reread theconfiguration file.
Unix System Administration & Programming30 January 2013 200
Sending Signals to Process
-
7/29/2019 LinuxOS 8th Sem
201/332
Syntax kill [-signal] pid(s)
kill [-signal] %jobID
Send the specified signal to a process
Default signal is TERM
killall
kill -l list all available signal
Unix System Administration & Programming30 January 2013 201
Altering Process Scheduling Priority
-
7/29/2019 LinuxOS 8th Sem
202/332
At process invocation timeSyntax
nice [-n adjustment] command
Process are scheduled with a defult pririty of 0
Priority value range (-20 highest to 19 = lowest) renice change the priority of a running process
renice # [ [-p|-g ] PID] [[-u] user]
# is the priority value Once the priority value is raised, a non-privileged
user cannot lower it
Unix System Administration & Programming30 January 2013 202
Linux Process Hierarchy[0]
-
7/29/2019 LinuxOS 8th Sem
203/332
Login shell
ChildChildChild
GrandchildGrandchild
[0]
Daemone.g. httpd
init [1]/etc/inittab
Unix System Administration & Programming30 January 2013 203
pstree
-
7/29/2019 LinuxOS 8th Sem
204/332
Kernel threads, not real processes
Forked processes for network connectionsUnix System Administration & Programming30 January 2013 204
Scheduling processes to execute Later
-
7/29/2019 LinuxOS 8th Sem
205/332
Syntax at time
atq [user]
atrm [user | atjobID]
Commands will be executed at the time indicated Non-redirected output is mailed to the user
Unix System Administration & Programming30 January 2013 205
Scheduling processes-cron
Many aspects of system administration require
-
7/29/2019 LinuxOS 8th Sem
206/332
Many aspects of system administration requirethings to be done on a routine basis
Rotating logs
building help files
checking disk space
checking permissions
Linux provides scheduling mechanism refereed toas cron.
Cron has two parts Daemon - crond
table of actions /etc/crontab
Unix System Administration & Programming30 January 2013 206
Cron
-
7/29/2019 LinuxOS 8th Sem
207/332
the crond Daemon is started at boot time the daemon wakes up every minute to check its
table of actions if their is something to do -> run command
if nothing to do --> go back to sleep for 1 min Cron table is a list (time,commnd) pairs. The
format isminute hour day month dayofweek command
Unix System Administration & Programming30 January 2013 207
Crontab
Commands can be scheduled by
-
7/29/2019 LinuxOS 8th Sem
208/332
Commands can be scheduled by
minute (0 59)
Hour ( 0 to 23)
Day of the month (1 - 31)
Month ( 1 to 12) Day of the week (0=Sunday 6 = sat, or use
mon,tues,wed)
* Matches everything
1-3 Matches range 1,5 Matches Series
Unix System Administration & Programming30 January 2013 208
Crontab
-
7/29/2019 LinuxOS 8th Sem
209/332
Example
01 * * * * command1 # hourly at 1 minute past
* 1 * * * command2 # daily at 1 am
04 1 1-5 * * command3 # run at 4 minute past 1 each day
Unix System Administration & Programming30 January 2013 209
Cron
-
7/29/2019 LinuxOS 8th Sem
210/332
Under Redhat Linux the cron table is used toexecute a set of commands in some specialdirectories /etc/cron.hourly
/etc/cron.daily contains logrotate, makewhatis,slocate,tmpwatch
/etc/cron.weekly
/etc/cron.monthly You can add you own commands to the appropriate directory,
but remember they need to be batch commands as they willrun automatically
Unix System Administration & Programming30 January 2013 210
crontab commands
-
7/29/2019 LinuxOS 8th Sem
211/332
crontab l List crontab e Edit
crontab l > cronfile
crontab cronfile
administrator can explicitly give permission tospecific users or exclude specific user by enteringtheir user identification in :
/etc/cron.d/cron.allow /etc/cron.d/cron.deny
Unix System Administration & Programming30 January 2013 211
Common Uses for CRON
-
7/29/2019 LinuxOS 8th Sem
212/332
Cleaning the filesystem Distribution of config files
Rotating log files
Backups
Unix System Administration & Programming30 January 2013 212
Shell scripts