Basics of the Unix/Linux

EnvironmentManipulating & Printing Files; Regular Expressions

CERI PrintersLong Building

3892_grad -- B & W printer in grad area3892_hpcolor -- Color printer in grad area3892_hpxlfp -- Poster printer in grad area

House 33876_langston -- B & W printer near Steve’s office3876_hpcolor -- Color printer near conference

room3876_grad – B & W duplex printer in Sun lab

House 23890_hpcolor – Color printer in copier room3890_copy – B & W printer in copier room

House 13904_tek -- Color printer 3904_hallway -- B & W printer

Printing Commandslpr: submit files for printing

%lpr -P3892_grad file.txt

lpq: show printer queue statususeful to find out if other jobs are before yours

%lpq -P3892_grad

3892_grad is ready and printing

Rank Owner Job File(s) Total Size

active hdeshon 146 junk.pdf 108544 bytes

Identifies the job

lprm: cancel print job%lprm -P3892_grad 146

lpstat: printer status informationuseful for finding out printer names on Macs, which

are not necessarily the same as on the Unix system

%lpstat –a

_3876langston accepting requests since Wed Aug 27 13:11:36 2008

hp_color_LaserJet_4600 accepting requests since Mon Aug 4 11:50:47 2008

More useful commandswc: word or line count

%wc suma1.hrdpicks

37753 253998 3561084 suma1.hrdpicks

Reports number of lines, words (separator=space), and characters in the file.

cmp: compare files

%cmp hw1.txt hw1a.txt

hw1.txt hw1a.txt differ: char 175, line 12

No output if the same, else reports byte and line numbers at which the first difference occurred (starts at 1).

diff: difference files

%diff hw1.txt hw1a.txt

12c12

< 2) [2] Create a directory in your account for this course - you might call it something like ESCI7205.

---

> 2) [2*] Create a directory in your account for this course - you might call it something like ESCI7205.

14c14

Sometimes useful (if files completely different is mess). < for file 1; > for file 2. (if have extra lines, will re-synch, afterwards.)

sort: alphabetical or numeric sort function

sort numerically%sort –n dirList | head –n3799799812

sort numerically and remove duplicates%sort –n -u dirList 799812825

• sort alphabetically% more samgps.datPELD -33.14318 -70.67493 CAP [5] 1993 1997 1998 1999 2002 CHILE OKRTCOGO -31.15343 -70.97526 CAP [3] 1993 1996 2002 CHILE OKRTMORA -30.20823 -70.78971 CAP [3] 1993 1996 2002 CHILE OKRTMOR2 -30.20823 -70.78971 CAP [?] CHILE OKRT. . .

% sort samgps.datABAC -24.433 -66.217 SAGA [-] ARGENTINA NORTABEL -25.667 -65.483 SAGA [-] ARGENTINA NORTACOL -30.78337 -66.21338 CAP [3] 1993 1997 2000 ARGENTINA OKRTACPM -33.447181 -70.537434 CAP2 [c] continuous (2005-) CHILEADLS -26.08449 -67.4191 CAP [2] 1993 1997 ARGENTINA OKRTAGAL -24.317 -66.467 SAGA [-] ARGENTINA NORT. . .

Alphabetically with numbers%sort dirList | head –n31045 1045 comes before 950 becomes 1 comes before 910461113

sort by column or position%head –n3 SUMA.NEW.loc1 1 1918 9 22 9 54 49.29 -1.698 98.298 15.0 0.0 0.0 ehb FEQ x2 2 1929 8 1 5 1 49.48 9.599 92.802 30.0 0.0 0.0 ehb FEQ x3 3 1929 12 9 6 49 57.79 4.003 94.545 20.0 0.0 0.0 ehb FEQ x

%sort –k9 –n SUMA.NEW.loc 23 23 1943 6 8 20 42 42.66 -2.938 101.747 30.0 0.0 0.0 ehb FEQ x376 376 1975 4 13 14 23 5.61 -2.511 99.975 30.0 4.9 0.0 ehb FEQ x465 465 1977 7 29 21 6 0.7 -2.466 99.870 30.0 5.1 0.0 ehb FEQ d

sort using a different separator (default is white space)%sort -n -t"," -k9 SUMA.NEW.loc.csv 1,1,1918,9,22,9,54,49.29,,,-1.698,,,98.298,,,15.0,,0.0,,0.0,ehb,FEQ,,x10,10,1935,11,25,10,3,7.39,,,,5.886,,,93.737,,,35.0,,0.0,,0.0,ehb,FEQ,,x100,100,1964,11,7,0,50,7.03,,,,1.801,,,99.483,,,15.0,,5.0,,0.0,ehb,FEQ,,x

• Read the man page to see what else it will do.

NAME sort - sort, merge, or sequence check text files

SYNOPSIS /usr/bin/sort [-bcdfimMnru] [-k keydef] [-o output] [-S kmem] [-t char] [-T directory] [ -y [kmem]] [-z recsz] [+pos1 [-pos2]] [file...]

/usr/xpg4/bin/sort [-bcdfimMnru] [-k keydef] [-o output] [-S kmem] [-t char] [-T directory] [ -y [kmem]] [-z recsz] [+pos1 [-pos2]] [file...]

From now on, you will be expected to read the man pages for all the commands

we have used or will use to see how to use them and what they will do.

Timecal: displays a calendar

default is current monthwill also display the yearok way to figure out julian day using the –j flag

%cal September 2009Su Mo Tu We Th Fr Sa 1 2 3 4 5 6 7 8 9 10 11 1213 14 15 16 17 18 1920 21 22 23 24 25 2627 28 29 30

% cal -j

September 2009 Su Mo Tu We Th Fr Sa 244 245 246 247 248249 250 251 252 253 254 255256 257 258 259 260 261 262263 264 265 266 267 268 269270 271 272 273

date: displays date and time

%dateWed Aug 27 17:12:01 CDT 2008

%date –u –r 10Thu Jan 1 00:00:10 UTC 1970

Basic Mathbc: basic math calculator

+, -, *, /, %, ^, sqrtalso test Boolean expressions and >,<,!=, etc.quit or CNTL-D to exit

expr: evaluate the expressionmore powerful, command line calculator for integer math

and string comparison

units: unit conversion

Job Control top: lists all processes currently running

ps: process status, another way to display process identification numbers (PID)

%ps –u hdeshon

UID PID TTY TIME CMD501 13255 ttys003 0:00.03 ssh -Y hdeshon@enigma501 13328 ttys006 0:00.13 gs -sDEVICE=x11 workinglocs.ps 0 13381 ttys006 0:00.00 ps -u hdeshon

kill: allows you to hard kill processes by PID%kill -9 13328

CNTL-Z: suspends the current job

bg: resume job but runs it in the backgroundthis functionality can be set on the command line by

adding an & to the end of the command/script

fg: resume job and runs it in the foreground

jobs: lists all jobs running in the background, including their PIDs

Finding/Searchingfind: search for files

Say I want to see if I have a file “Volcanoes.dat” in my “dem” and “bin” subdirectories.

%find ~/dem -name Volcanoes.dat

/gaia/home/rsmalley/dem/Volcanoes.dat

% find ~/bin -name "*olcanoes*"

%

OK, that’s nice, but not yet too useful (I could have cd’d into dem and done an ls, no need for a new command).

To make this really useful, we need a way to search for patterns in the filenames (or within files).

Enter Regular Expressions.

A regular expression is a set of characters that specify a pattern.

Regular Expressions If you master regular expressions, changing and search for

text becomes easy just from the command line.

Regular expressions are accepted input for grep, sed, awk, perl, vim and other unix commands.

Much like learning the shells, it is all about syntax & we’ll just scratch the surface here.

http://www.regular-expressions.info

We are going to have two kinds of special characters, or metacharacters.

Those that mean something special to the shell (such as the “$” on a shell or environment variable or the “/” in a path).

Those that are used to specify a pattern.

--------------------And will need a way to “turn off”, or escape, the special

meaning.

RE examples using grepgrep: search for a pattern

HIGHLY useful & it is worth your time to sit down with the man page

%grep DEQ SUMA.NEW.loc

31 31 1953 7 7 4 7 52.01 0.892 100.368 236.8 0.0 0.0 ehb DEQ d32 32 1953 11 13 16 17 13.21 3.964 96.037 62.7 0.0 0.0 ehb DEQ d

Basic RE symbols. : Matches a single character

%grep B…M SUMA.NEW.loc

799 1988 4 3 14 27 10.65 4.738 94.411 30.0 5.8 5.8 ehb BEQ Md

840 1990 1 22 17 26 12.96 3.913 96.063 46.0 6.0 5.9 ehb BEQ Md

* : Matches zero or more instances of the preceding character

%grep 'MC10*' dblocsat2.pf

23 MC1000 P 1208598406.00526 -1.000 d -1.00 -1.00 n -1.00 -1.00 n

26 MC1000 S 1208598407.86593 -1.000 d -1.00 -1.00 n -1.00 -1.00 n

% find ~/dem -name *olcanoes*

find: No match.

WHY?

^ : Represents the beginning of a line

file example IND.pha# 1918 9 22 9 54 49.29 -1.698 98.298 15.0 0.0 0 0 -0.05 1COC 274.71 1 P MAN 346.71 1 P ZKW 450.71 1 P # 1926 6 28 3 23 26.82 -0.128 101.514 15.0 0.0 0 0 1.17 2COC 303.18 1 P

%grep ^# IND.pha# 1918 9 22 9 54 49.29 -1.698 98.298 15.0 0.0 0 0 -0.05 1# 1926 6 28 3 23 26.82 -0.128 101.514 15.0 0.0 0 0 1.17 2

or

%grep –c ^# IND.pha the –c flag counts the number of matches8329

$ : Represents the end of the linefile example IND.pha# 1918 9 22 9 54 49.29 -1.698 98.298 15.0 0.0 0 0 -0.05 1COC 274.71 1 P MAN 346.71 1 P ZKW 450.71 1 P # 1926 6 28 3 23 26.82 -0.128 101.514 15.0 0.0 0 0 1.17 2COC 303.18 1 P

%grep ‘P_*$’ IND.pha | head –n2COC 274.71 1 P______ MAN 346.71 1 P______

or

%grep –c ‘P_*$’ IND.pha the –c flag counts the number of matches831857

\ : Tells the shell you are looking for a symbol%grep '\*' suma.stations | head –n2

* AGD +11.529000 +042.824000

* AIS -37.797000 +077.569000

[ ] : Matches members of the sets/ranges within the brackets

%grep '[DB]EQ' SUMA.NEW.loc

3478 2005 7 4 16 7 35.23 10.301 93.576 29.9 4.9 0.0 ehb DEQ Md

3480 2005 7 5 1 52 4.16 1.822 97.068 30.0 6.2 6.8 ehb BEQ Md

3481 2005 7 5 7 57 27.19 2.244 94.978 15.7 5.1 4.5 ehb DEQ Md

Non-printable charactersThe following syntax works with a range of

commands and programs that recognize regular expressions (sed, awk, perl, printf, etc)

\t : for a tab character \r : for carriage return \n : for line feed or new line. \s : for a white space

sed: powerful command line text editor

awk: powerful pattern-directed scanning and processing language

so powerful that we will devote a full week to it in the future

• Use find and grep in combination

%find ~/src | grep sachdr.c

/Users/hdeshon/src/sac_functions/getsachdr.c/Users/hdeshon/src/sac_functions/setsachdr.c