unix shell-scripting basics
TRANSCRIPT
5/14/2018 UNIX Shell-Scripting Basics - slidepdf.com
http://slidepdf.com/reader/full/unix-shell-scripting-basics-55a75931789f3 1/71
UNIX Shell-Scripting Basics
5/14/2018 UNIX Shell-Scripting Basics - slidepdf.com
http://slidepdf.com/reader/full/unix-shell-scripting-basics-55a75931789f3 2/71
Agenda
What is a shell? A shell script?
Introduction to bash
Running CommandsApplied Shell Programming
5/14/2018 UNIX Shell-Scripting Basics - slidepdf.com
http://slidepdf.com/reader/full/unix-shell-scripting-basics-55a75931789f3 3/71
What is a shell?
5/14/2018 UNIX Shell-Scripting Basics - slidepdf.com
http://slidepdf.com/reader/full/unix-shell-scripting-basics-55a75931789f3 4/71
What is a shell?
/bin/bash
5/14/2018 UNIX Shell-Scripting Basics - slidepdf.com
http://slidepdf.com/reader/full/unix-shell-scripting-basics-55a75931789f3 5/71
What is a shell?
#!/bin/bash
5/14/2018 UNIX Shell-Scripting Basics - slidepdf.com
http://slidepdf.com/reader/full/unix-shell-scripting-basics-55a75931789f3 6/71
What is a shell?
INPUT
shell
OUTPUT ERROR
5/14/2018 UNIX Shell-Scripting Basics - slidepdf.com
http://slidepdf.com/reader/full/unix-shell-scripting-basics-55a75931789f3 7/71
What is a shell?
Any Program But there are a few popular shells…
5/14/2018 UNIX Shell-Scripting Basics - slidepdf.com
http://slidepdf.com/reader/full/unix-shell-scripting-basics-55a75931789f3 8/71
Bourne Shells
/bin/sh
/bin/bash“Bourne- Again Shell”
Steve Bourne
5/14/2018 UNIX Shell-Scripting Basics - slidepdf.com
http://slidepdf.com/reader/full/unix-shell-scripting-basics-55a75931789f3 9/71
Other Common Shells
C Shell (/bin/csh)
Turbo C Shell (/bin/tcsh)
Korn Shell (/bin/ksh)
5/14/2018 UNIX Shell-Scripting Basics - slidepdf.com
http://slidepdf.com/reader/full/unix-shell-scripting-basics-55a75931789f3 10/71
An aside: What do I mean by /bin ?
C Shell (/bin/csh)
Turbo C Shell (/bin/tcsh)
Korn Shell (/bin/ksh)
5/14/2018 UNIX Shell-Scripting Basics - slidepdf.com
http://slidepdf.com/reader/full/unix-shell-scripting-basics-55a75931789f3 11/71
An aside: What do I mean by /bin ?
/bin, /usr/bin, /usr/local/bin
/sbin, /usr/sbin, /usr/local/sbin
/tmp
/dev /home/borwicjh
5/14/2018 UNIX Shell-Scripting Basics - slidepdf.com
http://slidepdf.com/reader/full/unix-shell-scripting-basics-55a75931789f3 12/71
What is a Shell Script?
A Text File
With Instructions
Executable
5/14/2018 UNIX Shell-Scripting Basics - slidepdf.com
http://slidepdf.com/reader/full/unix-shell-scripting-basics-55a75931789f3 13/71
What is a Shell Script?
% cat > hello.sh <<MY_PROGRAM
#!/bin/sh
echo „Hello, world‟ MY_PROGRAM
% chmod +x hello.sh
% ./hello.shHello, world
5/14/2018 UNIX Shell-Scripting Basics - slidepdf.com
http://slidepdf.com/reader/full/unix-shell-scripting-basics-55a75931789f3 14/71
What is a Shell Script? A Text File
% cat > hello.sh <<MY_PROGRAM
#!/bin/sh
echo „Hello, world‟ MY_PROGRAM
% chmod +x hello.sh
% ./hello.shHello, world
5/14/2018 UNIX Shell-Scripting Basics - slidepdf.com
http://slidepdf.com/reader/full/unix-shell-scripting-basics-55a75931789f3 15/71
An aside: Redirection
cat > /tmp/myfile
cat >> /tmp/myfile
cat 2> /tmp/myerr
cat < /tmp/myinput cat <<INPUTSome inputINPUT
cat > /tmp/x 2>&1
INPUT
env
OUTPUT ERROR
0
1 2
5/14/2018 UNIX Shell-Scripting Basics - slidepdf.com
http://slidepdf.com/reader/full/unix-shell-scripting-basics-55a75931789f3 16/71
What is a Shell Script? How To Run
% cat > hello.sh <<MY_PROGRAM
#!/bin/sh
echo „Hello, world‟ MY_PROGRAM
% chmod +x hello.sh
% ./hello.shHello, world
5/14/2018 UNIX Shell-Scripting Basics - slidepdf.com
http://slidepdf.com/reader/full/unix-shell-scripting-basics-55a75931789f3 17/71
What is a Shell Script? What To Do
% cat > hello.sh <<MY_PROGRAM
#!/bin/sh
echo „Hello, world‟ MY_PROGRAM
% chmod +x hello.sh
% ./hello.shHello, world
5/14/2018 UNIX Shell-Scripting Basics - slidepdf.com
http://slidepdf.com/reader/full/unix-shell-scripting-basics-55a75931789f3 18/71
What is a Shell Script? Executable
% cat > hello.sh <<MY_PROGRAM
#!/bin/sh
echo „Hello, world‟ MY_PROGRAM
% chmod +x hello.sh
% ./hello.shHello, world
5/14/2018 UNIX Shell-Scripting Basics - slidepdf.com
http://slidepdf.com/reader/full/unix-shell-scripting-basics-55a75931789f3 19/71
What is a Shell Script? Running it
% cat > hello.sh <<MY_PROGRAM
#!/bin/sh
echo „Hello, world‟ MY_PROGRAM
% chmod +x hello.sh
% ./hello.shHello, world
5/14/2018 UNIX Shell-Scripting Basics - slidepdf.com
http://slidepdf.com/reader/full/unix-shell-scripting-basics-55a75931789f3 20/71
Finding the program: PATH
% ./hello.sh
echo vs. /usr/bin/echo
% echo $PATH/bin:/usr/bin:/usr/local/bin:/home/borwicjh/bin
% which echo/usr/bin/echo
5/14/2018 UNIX Shell-Scripting Basics - slidepdf.com
http://slidepdf.com/reader/full/unix-shell-scripting-basics-55a75931789f3 21/71
Variables and the Environment
% hello.sh
bash: hello.sh: Command notfound
% PATH=“$PATH:.”
% hello.sh
Hello, world
5/14/2018 UNIX Shell-Scripting Basics - slidepdf.com
http://slidepdf.com/reader/full/unix-shell-scripting-basics-55a75931789f3 22/71
An aside: Quoting
% echo „$USER‟ $USER% echo “$USER”
borwicjh% echo “\”” ” % echo “deacnet\\sct”
deacnet\sct% echo „\”‟ \”
5/14/2018 UNIX Shell-Scripting Basics - slidepdf.com
http://slidepdf.com/reader/full/unix-shell-scripting-basics-55a75931789f3 23/71
Variables and the Environment
% env[…variables passed to sub-programs…] % NEW_VAR=“Yes”
% echo $NEW_VARYes% env[…PATH but not NEW_VAR…]
% export NEW_VAR% env[…PATH and NEW_VAR…]
5/14/2018 UNIX Shell-Scripting Basics - slidepdf.com
http://slidepdf.com/reader/full/unix-shell-scripting-basics-55a75931789f3 24/71
Welcome to Shell Scripting!
5/14/2018 UNIX Shell-Scripting Basics - slidepdf.com
http://slidepdf.com/reader/full/unix-shell-scripting-basics-55a75931789f3 25/71
How to Learn
manman bash
man cat
man man man –kman –k manual
Learning the Bash Shell , 2nd Ed.
“Bash Reference” Cards
http://www.tldp.org/LDP/abs/html/
5/14/2018 UNIX Shell-Scripting Basics - slidepdf.com
http://slidepdf.com/reader/full/unix-shell-scripting-basics-55a75931789f3 26/71
Introduction to bash
5/14/2018 UNIX Shell-Scripting Basics - slidepdf.com
http://slidepdf.com/reader/full/unix-shell-scripting-basics-55a75931789f3 27/71
Continuing Lines: \
% echo This \
Is \
A \
Very \
Long \
Command Line
This Is A Very Long Command Line
%
5/14/2018 UNIX Shell-Scripting Basics - slidepdf.com
http://slidepdf.com/reader/full/unix-shell-scripting-basics-55a75931789f3 28/71
Exit Status
$?
0 is True
% ls /does/not/exist
% echo $?
1
% echo $?
0
5/14/2018 UNIX Shell-Scripting Basics - slidepdf.com
http://slidepdf.com/reader/full/unix-shell-scripting-basics-55a75931789f3 29/71
Exit Status: exit
% cat > test.sh <<_TEST_
exit 3
_TEST_% chmod +x test.sh
% ./test.sh
% echo $?3
5/14/2018 UNIX Shell-Scripting Basics - slidepdf.com
http://slidepdf.com/reader/full/unix-shell-scripting-basics-55a75931789f3 30/71
Logic: test
% test 1 -lt 10
% echo $?
0% test 1 == 10
% echo $?
1
5/14/2018 UNIX Shell-Scripting Basics - slidepdf.com
http://slidepdf.com/reader/full/unix-shell-scripting-basics-55a75931789f3 31/71
Logic: test
test
[ ]
[ 1 –lt 10 ]
[[ ]]
[[ “this string” =~ “this” ]]
(( ))
(( 1 < 10 ))
5/14/2018 UNIX Shell-Scripting Basics - slidepdf.com
http://slidepdf.com/reader/full/unix-shell-scripting-basics-55a75931789f3 32/71
Logic: test
[ -f /etc/passwd ]
[ ! –f /etc/passwd ]
[ -f /etc/passwd –a –f /etc/shadow ]
[ -f /etc/passwd –o –f /etc/shadow ]
5/14/2018 UNIX Shell-Scripting Basics - slidepdf.com
http://slidepdf.com/reader/full/unix-shell-scripting-basics-55a75931789f3 33/71
An aside: $(( )) for Math
% echo $(( 1 + 2 ))
3
% echo $(( 2 * 3 ))6
% echo $(( 1 / 3 ))
0
5/14/2018 UNIX Shell-Scripting Basics - slidepdf.com
http://slidepdf.com/reader/full/unix-shell-scripting-basics-55a75931789f3 34/71
Logic: if
if something
then
:
# “elif” a contraction of “else if”:
elif something-else
then
:
else
then
:
fi
5/14/2018 UNIX Shell-Scripting Basics - slidepdf.com
http://slidepdf.com/reader/full/unix-shell-scripting-basics-55a75931789f3 35/71
Logic: if
if [ $USER –eq “borwicjh” ]
then
:
# “elif” a contraction of “else if”:
elif ls /etc/oratab
then
:
else
then
:
fi
5/14/2018 UNIX Shell-Scripting Basics - slidepdf.com
http://slidepdf.com/reader/full/unix-shell-scripting-basics-55a75931789f3 36/71
Logic: if
# see if a file exists
if [ -e /etc/passwd ]
thenecho “/etc/passwd exists”
else
echo “/etc/passwd not found!” fi
5/14/2018 UNIX Shell-Scripting Basics - slidepdf.com
http://slidepdf.com/reader/full/unix-shell-scripting-basics-55a75931789f3 37/71
Logic: for
for i in 1 2 3
do
echo $idone
5/14/2018 UNIX Shell-Scripting Basics - slidepdf.com
http://slidepdf.com/reader/full/unix-shell-scripting-basics-55a75931789f3 38/71
Logic: for
for i in /*
do
echo “Listing $i:” ls -l $i
read
done
5/14/2018 UNIX Shell-Scripting Basics - slidepdf.com
http://slidepdf.com/reader/full/unix-shell-scripting-basics-55a75931789f3 39/71
Logic: for
for i in /*
do
echo “Listing $i:” ls -l $i
read
done
5/14/2018 UNIX Shell-Scripting Basics - slidepdf.com
http://slidepdf.com/reader/full/unix-shell-scripting-basics-55a75931789f3 40/71
Logic: for
for i in /*
do
echo “Listing $i:”
ls -l $i
read
done
5/14/2018 UNIX Shell-Scripting Basics - slidepdf.com
http://slidepdf.com/reader/full/unix-shell-scripting-basics-55a75931789f3 41/71
Logic: C-style for
for (( expr1 ;
expr2 ;
expr3 ))
do
listdone
5/14/2018 UNIX Shell-Scripting Basics - slidepdf.com
http://slidepdf.com/reader/full/unix-shell-scripting-basics-55a75931789f3 42/71
Logic: C-style for
LIMIT=10
for (( a=1 ;
a<=LIMIT ;
a++ ))
do
echo –n “$a ” done
5/14/2018 UNIX Shell-Scripting Basics - slidepdf.com
http://slidepdf.com/reader/full/unix-shell-scripting-basics-55a75931789f3 43/71
Logic: while
while something
do
:
done
5/14/2018 UNIX Shell-Scripting Basics - slidepdf.com
http://slidepdf.com/reader/full/unix-shell-scripting-basics-55a75931789f3 44/71
Logic: while
a=0; LIMIT=10
while [ "$a" -lt "$LIMIT" ]
do
echo -n "$a ”
a=$(( a + 1 ))
done
5/14/2018 UNIX Shell-Scripting Basics - slidepdf.com
http://slidepdf.com/reader/full/unix-shell-scripting-basics-55a75931789f3 45/71
Counters
COUNTER=0
while [ -e “$FILE.COUNTER” ]
do
COUNTER=$(( COUNTER + 1))
done
Note: race condition
5/14/2018 UNIX Shell-Scripting Basics - slidepdf.com
http://slidepdf.com/reader/full/unix-shell-scripting-basics-55a75931789f3 46/71
Reusing Code: “Sourcing”
% cat > /path/to/my/passwords <<_PW_
FTP_USER=“sct”
_PW_
% echo $FTP_USER
% . /path/to/my/passwords
% echo $FTP_USER
sct
%
5/14/2018 UNIX Shell-Scripting Basics - slidepdf.com
http://slidepdf.com/reader/full/unix-shell-scripting-basics-55a75931789f3 47/71
Variable Manipulation
% FILEPATH=/path/to/my/output.lis% echo $FILEPATH
/path/to/my/output.lis
% echo ${FILEPATH%.lis}/path/to/my/output
% echo ${FILEPATH#*/}
path/to/my/output.lis
% echo ${FILEPATH##*/}
output.lis
5/14/2018 UNIX Shell-Scripting Basics - slidepdf.com
http://slidepdf.com/reader/full/unix-shell-scripting-basics-55a75931789f3 48/71
It takes a long time to become a bash guru…
5/14/2018 UNIX Shell-Scripting Basics - slidepdf.com
http://slidepdf.com/reader/full/unix-shell-scripting-basics-55a75931789f3 49/71
Running Programs
5/14/2018 UNIX Shell-Scripting Basics - slidepdf.com
http://slidepdf.com/reader/full/unix-shell-scripting-basics-55a75931789f3 50/71
Reasons for Running Programs
Check Return Code
$?
Get Job Output
OUTPUT=`echo “Hello”`
OUTPUT=$(echo “Hello”)
Send Output Somewhere
Redirection: <, >
Pipes
5/14/2018 UNIX Shell-Scripting Basics - slidepdf.com
http://slidepdf.com/reader/full/unix-shell-scripting-basics-55a75931789f3 51/71
Pipes
Lots of Little Tools
echo “Hello” | \
wc -c
INPUT
echo
OUTPUT ERROR
0
1 2
INPUT
wc
OUTPUT ERROR
0
1 2
A Pipe!
5/14/2018 UNIX Shell-Scripting Basics - slidepdf.com
http://slidepdf.com/reader/full/unix-shell-scripting-basics-55a75931789f3 52/71
Email Notification
% echo “Message” | \
mail –s “Here‟s your message” \
5/14/2018 UNIX Shell-Scripting Basics - slidepdf.com
http://slidepdf.com/reader/full/unix-shell-scripting-basics-55a75931789f3 53/71
Dates
% DATESTRING=`date +%Y%m%d`
% echo $DATESTRING
20060125
% man date
5/14/2018 UNIX Shell-Scripting Basics - slidepdf.com
http://slidepdf.com/reader/full/unix-shell-scripting-basics-55a75931789f3 54/71
FTP the Hard Way
ftp –n –u server.wfu.edu <<_FTP_
user username password
put FILE
_FTP_
5/14/2018 UNIX Shell-Scripting Basics - slidepdf.com
http://slidepdf.com/reader/full/unix-shell-scripting-basics-55a75931789f3 55/71
FTP with wget
wget \ftp://user:[email protected]/file
wget –r \ftp://user:[email protected]/dir/
5/14/2018 UNIX Shell-Scripting Basics - slidepdf.com
http://slidepdf.com/reader/full/unix-shell-scripting-basics-55a75931789f3 56/71
FTP with curl
curl –T upload-file \
-u username:password \
ftp://server.wfu.edu/dir/file
5/14/2018 UNIX Shell-Scripting Basics - slidepdf.com
http://slidepdf.com/reader/full/unix-shell-scripting-basics-55a75931789f3 57/71
Searching: grep
% grep rayra /etc/passwd
% grep –r rayra /etc
% grep –r RAYRA /etc
% grep –ri RAYRA /etc
% grep –rli rayra /etc
5/14/2018 UNIX Shell-Scripting Basics - slidepdf.com
http://slidepdf.com/reader/full/unix-shell-scripting-basics-55a75931789f3 58/71
Searching: find
% find /home/borwicjh \
-name „*.lis‟
[all files matching *.lis]
% find /home/borwicjh \
-mtime -1 –name „*.lis‟
[*.lis, if modified within 24h] % man find
5/14/2018 UNIX Shell-Scripting Basics - slidepdf.com
http://slidepdf.com/reader/full/unix-shell-scripting-basics-55a75931789f3 59/71
Searching: locate
% locate .lis
[files with .lis in path]
% locate log
[also finds “/var/log/messages”]
5/14/2018 UNIX Shell-Scripting Basics - slidepdf.com
http://slidepdf.com/reader/full/unix-shell-scripting-basics-55a75931789f3 60/71
Applied Shell Programming
5/14/2018 UNIX Shell-Scripting Basics - slidepdf.com
http://slidepdf.com/reader/full/unix-shell-scripting-basics-55a75931789f3 61/71
Make Your Life Easier
TAB completion
Control+R
history
cd -
Study a UNIX Editor
5/14/2018 UNIX Shell-Scripting Basics - slidepdf.com
http://slidepdf.com/reader/full/unix-shell-scripting-basics-55a75931789f3 62/71
pushd/popd
% cd /tmp% pushd /var/log
/var/log /tmp
% cd ..% pwd
/var
% popd/tmp
5/14/2018 UNIX Shell-Scripting Basics - slidepdf.com
http://slidepdf.com/reader/full/unix-shell-scripting-basics-55a75931789f3 63/71
Monitoring processes
ps
ps –ef
ps –u oracle
ps –C sshd
man ps
5/14/2018 UNIX Shell-Scripting Basics - slidepdf.com
http://slidepdf.com/reader/full/unix-shell-scripting-basics-55a75931789f3 64/71
“DOS” Mode Files
#!/usr/bin/bash^M
FTP transfer in ASCII, or
dos2unix infile > outfile
5/14/2018 UNIX Shell-Scripting Basics - slidepdf.com
http://slidepdf.com/reader/full/unix-shell-scripting-basics-55a75931789f3 65/71
sqlplus
JOB=“ZZZTEST”
PARAMS=“ZZZTEST_PARAMS”
PARAMS_USER=“BORWICJH”
sqlplus $BANNER_USER/$BANNER_PW << _EOF_
set serveroutput on
set sqlprompt ""
EXECUTE WF_SATURN.FZ_Get_Parameters('$JOB','$PARAMS', '$PARAMS_USER');
_EOF_
5/14/2018 UNIX Shell-Scripting Basics - slidepdf.com
http://slidepdf.com/reader/full/unix-shell-scripting-basics-55a75931789f3 66/71
sqlplus
sqlplus $USER/$PASS @$FILE_SQL \$ARG1 $ARG2 $ARG3
if [ $? –ne 0 ]then
exit 1fiif [ -e /file/sql/should/create ]then[…use SQL- created file…]
fi
Ask Amy Lamy!
5/14/2018 UNIX Shell-Scripting Basics - slidepdf.com
http://slidepdf.com/reader/full/unix-shell-scripting-basics-55a75931789f3 67/71
Passing Arguments
% cat > test.sh <<_TEST_
echo “Your name is \$1 \$2”
_TEST_
% chmod +x test.sh
% ./test.sh John Borwick ignore-this
Your name is John Borwick
5/14/2018 UNIX Shell-Scripting Basics - slidepdf.com
http://slidepdf.com/reader/full/unix-shell-scripting-basics-55a75931789f3 68/71
INB Job Submission Template
$1: user ID$2: password
$3: one-up number
$4: process name$5: printer name
% /path/to/your/script $UI $PW \$ONE_UP $JOB $PRNT
5/14/2018 UNIX Shell-Scripting Basics - slidepdf.com
http://slidepdf.com/reader/full/unix-shell-scripting-basics-55a75931789f3 69/71
Scheduling Jobs
% crontab -l
0 0 * * * daily-midnight-job.sh
0 * * * * hourly-job.sh
* * * * * every-minute.sh
0 1 * * 0 1AM-on-sunday.sh
% EDITOR=vi crontab –e% man 5 crontab
5/14/2018 UNIX Shell-Scripting Basics - slidepdf.com
http://slidepdf.com/reader/full/unix-shell-scripting-basics-55a75931789f3 70/71