unit 4 scripting and the shell

Scripting and the Shell

Prepared By

Prof. Bhushan Pawar

www.bhushanpawar.com

SHELL BASICS

• All major shells in the sh(including Bourne shell(Bash),ksh(KornShell).

• Command for editing :-– emacs commands– <Control-E> goes to the end of the line– <Control-A> to the beginning.– <Control-P> steps backward through recently

executed commands and recalls them for editing.– <Control-R> searches incrementally through your

history to find old commands.– http://www.computerhope.com/unix/uemacs.htm

Prof.Bhushan Pawar www.bhushanpawar.com

SHELL BASICS (Continue…)

• If you like vi, put your shell’s command-line editing into vi mode like this:

$ set -o vi

$ set -o emacs


Pipes and redirection

• Every process has at least threecommunication channels available to it:

– “standard input” (STDIN)

– “standard output”(STDOUT)

– “standard error” (STDERR).


Pipes and redirection(Continue…)

• UNIX has a unified I/O model in which eachchannel is named with a small integer called a(File Descriptor).

• The exact number assigned to a channel is notusually significant, but STDIN, STDOUT, andSTDERR are guaranteed to correspond to filedescriptors 0, 1, and 2, so it’s safe to refer tothese channels by number. In the context of aninteractive terminal window, STDIN normallyreads from the keyboard and both STDOUT andSTDERR write their output to the screen.



• The shell interprets the symbols <, >, and >>as instructions to reroute a command’s inputor output to or from a file. A < symbolconnects the command’s STDIN to thecontents of an existing file. The > and >>symbols redirect STDOUT.

• > replaces the file’s existing contents, and >>appends to them.



• $ echo "This is a test message." > /tmp/mymessage

-it stores a single line in the file /tmp/mymessage,creating the file if necessary.

• To connect the STDOUT of one command to the STDIN of another, use the | symbol, commonly known as a pipe.

– E.g. $ ps -ef | grep httpd

(example runs ps to generate a list of processes and pipes it through the grep command to select lines that contain the

word httpd.)


Variables and quoting

• Variable names are unmarked in assignments but prefixed with a dollar sign ($)when their values are referenced.

– E.g. $ etcdir='/etc‘

$ echo $etcdir


Common filter commands

• cut: separate lines into fields

– Syntax:- cut -c4 file.txt

– Explanation : The above cut command prints the fourth character in each line of the file

– ( http://www.folkstalk.com/2012/02/cut-command-in-unix-linux-examples.html )


http://www.folkstalk.com/2012/02/cut-command-in-unix-linux-examples.html

sort command

• Sort: Sort command in unix or linux system is usedto order the elements or text. Sort command has thecapability of sorting numerical values and strings.

– Syntax :- sort [options] filename

– (http://www.folkstalk.com/2012/08/sort-command-examples-in-unix-linux.html )


http://www.folkstalk.com/2012/08/sort-command-examples-in-unix-linux.html

sort options

• -b : Ignores leading spaces in each line • -d : Uses dictionary sort order. Conisders only spaces and

alphanumeric characters in sorting • -f : Uses case insensitive sorting. • -M : Sorts based on months. Considers only first 3 letters as month.

Eg: JAN, FEB • -n : Uses numeric sorting • -R : Sorts the input file randomly. • -r : Reverse order sorting • -k : Sorts file based on the data in the specified field positions. • -u : Suppresses duplicate lines • -t : input field separator


Uniq command

• Uniq command in unix or linux system is used to suppress the duplicate lines from a file

• Syntax:- uniq [option] filename

– (http://www.folkstalk.com/search?q=uniq+command+in+%3Blinux )


http://www.folkstalk.com/search?q=uniq+command+in+;linux

uniq options

• c : Count of occurrence of each line.

• d : Prints only duplicate lines.

• D : Print all duplicate lines

• f : Avoid comparing first N fields.

• i : Ignore case when comparing.

• s : Avoid comparing first N characters.

• u : Prints only unique lines.

• w : Compare no more than N characters in lines


wc command

• wc command in unix or linux is used to find the number of lines, words and characters in a file.

• Syntax :- wc [options] filenames

• ( http://www.folkstalk.com/2012/07/wc-command-examples-in-unix-linux.html#more )


http://www.folkstalk.com/2012/07/wc-command-examples-in-unix-linux.html

wc options

• -l : Prints the number of lines in a file.

• -w : prints the number of words in a file.

• -c : Displays the count of bytes in a file.

• -m : prints the count of characters from a file.

• -L : prints only the length of the longest line in a file.


other commands

• tee

• head

• tail

• grep


tee command

• tee :- tee command writes to the STDOUT, and toa file at a time .

• The following command writes the output only tothe file and not to the screen.– $ ls > filename

• The following command (with the help of teecommand) writes the output both to the screen(stdout) and to the file.– $ ls | tee filename(http://linux.101hacks.com/unix/tee-command-

examples/ )


http://linux.101hacks.com/unix/tee-command-examples/

head command

• The head command reads the first few lines ofany text given to it as an input and writesthem to standard output (which, by default, isthe display screen).

• head's basic syntax is:

– head [options] [file(s)]

( http://www.linfo.org/head.html )


http://www.linfo.org/head.html

Options of head command

• -c, --bytes=[-]N – print the first N bytes of each file; with the leading '-', print all but the

last N bytes of each file

• -n, --lines=[-]N – print the first N lines instead of the first 10; with the leading '-', print

all but the last N lines of each file

• -q, --quiet, --silent – never print headers giving file names

• -v, --verbose – always print headers giving file names


tail command

• tail :-output the last part of files

• Syntax :-

– tail [OPTION]... [FILE]...

(http://linux.about.com/library/cmd/blcmdl1_tail.htm )


http://linux.about.com/library/cmd/blcmdl1_tail.htm

Options of tail command

• -c, --bytes=N– output the last N bytes

• -f, --follow[={name|descriptor}] – output appended data as the file grows; -f, --

follow, and --follow=descriptor are equivalent

• -F– same as --follow=name --retry

• -n, --lines=N– output the last N lines, instead of the last 10


grep command

• grep :- The grep command allows you to search one file or multiple files for lines that contain a pattern. Exit status is 0 if matches were found, 1 if no matches were found, and 2 if errors occurred.

• Syntax– grep [options] pattern [files]

(http://www.techonthenet.com/unix/basic/grep.php )


http://www.techonthenet.com/unix/basic/grep.php

Options of grep command


Bash Scripting

• Comment start with hash mark (#) and continue to the end of line.

#!/bin/bash

echo "Hello, world!“

– The first line is known as the “shebang” statementand declares the text file to be a script forinterpretation by /bin/bash.


Bash Scripting (Continue…)

• The kernel looks for this syntax when deciding

how to execute the file. From the perspective

of the shell spawned to execute the script, the

shebang line is just a comment.

• If bash were in a different location, we need to

adjust this line


Bash Scripting (Continue…)

• To prepare the file for running, just turn on its execute bit

$ chmod +x helloworld

$ ./helloworld– Hello, world!

• You can also invoke the shell as an interpreter directly:

$ bash helloworld– Hello, world!

$ source helloworld– Hello, world!

• The first command runs helloworld in a new instance of bash, and the second makes your existing login shell read and execute the contents of the file


Regular Expression

• Regular expressions are powerful, but they cannot recognize all possible grammars.

• They are so common that the name is usually shortened to “regex.”


The matching process

• Code that evaluates a regular expression attempts tomatch a single given text string to a single givenpattern. The “text string” to match can be very longand can contain embedded newlines.

• For the matcher to declare success, the entire searchpattern must match a contiguous section of the searchtext. However, the pattern can match at any position.

• After a successful match, the evaluator returns the textof the match along with a list of matches for anyspecially delimited subsections of the pattern.


Literal characters

• In general, characters in a regular expressionmatch themselves. So the pattern

I am the walrus

• Matches the string “I am the walrus” and thatstring only. Since it can match anywhere in thesearch text, the pattern can be successfullymatched to the string “I am the egg man. I amthe walrus. Koo koo ka-choo!” However, theactual match is limited to the “I am the walrus”portion. Matching is case sensitive.


Special characters

http://perldoc.perl.org/perlrecharclass.htmlProf.Bhushan Pawar www.bhushanpawar.com

Example

I am the (walrus|egg man)\.matches either “I am the walrus.” or “I am the egg man.”. This example also demonstrates escaping of special characters (here, the dot). The pattern

(I am the (walrus|egg man)\. ?){1,2}matches any of the following:

• I am the walrus.• I am the egg man.• I am the walrus. I am the egg man.• I am the egg man. I am the walrus.


PERL PROGRAMMING

• Perl created by Larry Wall

• More power than bash

• Perl does not impose much stylistic discipline on developers, so Perl code written without regard for readability can be cryptic. Perl has been accused of being a write-only language

• Perl’s catch phrase is that “there’s more than one way to do it.”


PERL PROGRAMMING(Continue…)

• Perl statements are separated by semicolons.Comments start with a hash mark (#) and continue to the end of the line. Blocks of statements are enclosed in curly braces.


Program

#!/usr/bin/perl

print "Hello, world!\n";

• As with bash programs, you must either chmod +x the executable file or invoke the Perl interpreter directly.


$ ./helloworld

Hello, world!


Variables and arrays

• three fundamental data types

– Scalars (that is, unitary values such as numbers and strings)

– Arrays

– Hashes (also known as associative Arrays)

• Scalar variables start with $, Array variables start with @, and Hash variables start with %


Variables and arrays(Continue…)

• In Perl, the terms “list” and “array” are oftenused interchangeably, but it’s more accurate tosay that a list is a series of values and an array isa variable that can hold such a list.

• The individual elements of an array are scalars, solike ordinary scalar variables, their names beginwith $. Array subscripting begins at zero, and theindex of the highest element in array

@a is $#a.• The array @ARGV contains the script’s command

line arguments.


Program

#!/usr/bin/perl

@items = ("socks", "shoes", "shorts");

printf "There are %d articles of clothing.\n",$#items + 1;

print "Put on ${items[2]} first, then ", join(" and ", @items[0,1]), ".\n";

• The output:

$ perl clothes

There are 3 articles of clothing.

Put on shorts first, then socks and shoes.


Explanation of the program

• Array and string literals

– Perl does not strictly require that all strings be quoted so @items = (socks, shoes, shorts); is also same

• Function calls

– Both print and printf accept an arbitrary number of arguments, and the arguments are separated by commas



• join receives three string arguments: “ and ”,“socks”, and “shoes”. It concatenates its secondand subsequent arguments, inserting a copy ofthe first argument between each pair. The resultis “socks and shoes”.



• Type conversions in expressions

The magic is in the + operator, which alwaysimplies arithmetic. It converts its arguments tonumbers and produces a numeric result. Similarly,the dot operator (.), which concatenates strings,

converts its operands as needed: "2" . (12 ** 2) yields“2144”.


Regular expressions in Perl

• Regular expressions in Perl by “binding” strings to regex operations with the

=~ operator.

e.g if ($text =~ m/ab+c/) { }

• checks to see whether the string stored in $text matches the regular expression ab+c.

• To operate on the default string, $_ , you can simply omit the variable name and binding operator.

• In fact, you can omit the m, too, since the operation defaults to matching


Program

#!/usr/bin/perl$names = "huey dewey louie";$regex = '(\w+)\s+(\w+)\s+(\w+)';if ($names =~ m/$regex/) {print "1st name is $1.\n2nd name is $2.\n3rd name is$3.\n";$names =~ s/$regex/\2 \1/;print "New names are \"${names}\".\n";} else {print qq{"$names" did not match "$regex".\n};}

(http://www.tutorialspoint.com/perl/perl_qq.htm)


Output

The output:

$ perl testregex

1st name is huey.

2nd name is dewey.

3rd name is louie.

New names are "dewey huey".


Input and output

• When you open a file for reading or writing, you define a “filehandle” to identify the channel.

• INFILE & OUTFILE are the filehandlers.

• The while loop condition is <INFILE>


Program

#!/usr/bin/perl

open(INFILE, "</etc/passwd") or die "Couldn’t open /etc/passwd";

open(OUTFILE, ">/tmp/passwd") or die "Couldn’t open /tmp/passwd";

while (<INFILE>)

{

($name, $pw, $uid, $gid, $gecos, $path, $sh) = split /:/;

print OUTFILE "$uid\t$name\n";

}


Continue…


PYTHON SCRIPTING

• Python was created by Guido van Rossum

• It’s easier to code and more readable than Perl.Python offers a simple-to-understand syntax thatis easy to follow even if you didn’t develop thecode. If you’re tired of remembering whichcomparison operators to use, you’ll appreciatePython’s unified approach.

• Python also offers additional data types thatsome system administrators find useful.


Simple Program

#!/usr/bin/python

print "Hello, world!“

Invoke the python interpreter directly:


$ ./helloworld

Hello, world!


sys — (System-specific parameters and functions)

• import sys

(https://docs.python.org/2/library/sys.html)


(https:/docs.python.org/2/library/sys.html

Program#!/usr/bin/python

name = 'Gwen'

rating = 10

characters = [ 'SpongeBob', 'Patrick', 'Squidward' ]

elements = ( 'lithium', 'carbon', 'boron' )

print "name:\t%s\nrating:\t%d" % (name, rating)

print "characters:\t%s" % characters

print "elements:\t%s" % (elements, )

This example produces the following output:

$ python objects

name: Gwen

rating: 10

characters: ['SpongeBob', 'Patrick', 'Squidward']

elements: ('lithium', 'carbon', 'boron')Prof.Bhushan Pawar

www.bhushanpawar.com

Loops• The fragment below uses a for…in construct to

iterate through the range 1 to 10.

for counter in range(1, 10):

print counter,

Output:

1 2 3 4 5 6 7 8 9


Continue…

• Both for and while loops can have else clausesat the end. The else clause is executed only ifthe loop terminates normally, as opposed toexiting through a break statement.

• This feature may initially seemcounterintuitive, but it handles certain usecases quite elegantly.


Any Question???• If you having any doubt then you can freely

ask me question on

[email protected]

Or

contact me on (+91)-7588318728


mailto:[email protected]

unit 4 scripting and the shell

Engineering