you can do it! start using perl to handle your voyager needs

You Can Do It!Start Using Perl to

Handle Your Voyager Needs.

Some Perl nomenclature

PERL – Practical Extraction and Report PERL – Practical Extraction and Report LanguageLanguage

(camel by O’Reilly)



PERL – Pathologically Eclectic Rubbish Lister PERL – Pathologically Eclectic Rubbish Lister (not really)(not really)




PERL – Pathologically Eclectic Rubbish Lister PERL – Pathologically Eclectic Rubbish Lister (not really)(not really)

TMTOWTDI – There’s More Than One Way To TMTOWTDI – There’s More Than One Way To Do ItDo It


Some Perl attributes

it’s a scripted language, not compiled -it’s a scripted language, not compiled -

faster, easier developmentfaster, easier development

runs plenty fast for most thingsruns plenty fast for most things

Some Perl attributes

it’s a scripted language, not compiled -it’s a scripted language, not compiled -

faster, easier developmentfaster, easier development

runs plenty fast for most thingsruns plenty fast for most things

Loose variable typing -Loose variable typing -

both good and bad,both good and bad,

but mostly goodbut mostly good

Your first program

#!/usr/local/bin/perl

print "Hello, World\n";

“Protecting” your program (Unix)

By default, your program is not executable.

chmod 744 your_program

You can execute it as owner of the file, anyone else can only read it.

Variables

$name

can be text or number: a character,a whole page of text,or any kind of number

context determines type

can go “both” ways

Variables, array of

@employee

Array of $employee variables

$employee[0]

$employee[1]

etc.

Variables, hash of

$lib{‘thisone’} = “2 days”;

$lib{‘thatone’} = “5 days”;

Thus can use

$grace_period = $lib{$libname}

when $libname is thatone,

$grace_period is 5 days

Variables, list of

($var1, $var2, $var3) =function_that_does_something;

This function returns a list of elements.

A list is always inside parentheses ().

Variables, assigning a value to

$var = value or expression

$array[n] = something;

@array = (); # empty array

%hash = (); # empty hash

Can be done almost anywhere, anytime.

use strict;

Requires that you declare all variables like this:

my $var;

my $var = something;

my @array = ();

Also makes Perl check your code.

Best Practices!

Variable scope, and good practices

use strict;

my $var;


my @array = ();

A variable declared like this is visible throughout your program.

Best Practices!



use strict;

my $var;


my @array = ();

A “my” declaration within code grouped within { and } is visible only in that section of code; it does not exist elsewhere.

Best Practices!

Scope: where in a program a variable exists.

$_ default parameter for many functions

$. current record/line number in current file

$/ input record separator (usually the newline character)

$, print() function output separator (normally anempty string)

$0 name of the Perl script being executed

$^T time, in seconds, when the script begins running

$^X full pathname of the Perl interpreter running the current script

some Special Variables

@ARGV array which contains the list of the commandline arguments

@Inc array which contains the list of directorieswhere Perl can look for scripts to execute(for use DBI and other modules)

%ENV hash variable which contains entries for yourcurrent environment variables


STDIN read from the standard input file handle(normally the keyboard)

STDOUT send output to the standard output file handle(normally the display)

STDERR send error output to the standard error filehandle (normally the display)

DATA file handle referring to any data following __END__

and dozens more…


String manipulation & other stuffGiven

$stuff = “this is me”;

These are not equivalent:

“print $stuff”

‘print $stuff’

`print $stuff`

Given



“print $stuff” is “print this is me”

‘print $stuff’

`print $stuff`

String manipulation & other stuff

Given



“print $stuff” is “print this is me”

‘print $stuff’ is ‘print $stuff’

`print $stuff`


Given $stuff = “this is me”;

`print $stuff` would have the operating system try to execute the command <print this is me>


This form should be used as$something = `O.S. command`

Example: $listing = ‘ls *.pl`;

The output of this ls command is placed, as possibly a large string, into the variable $listing. This syntax allows powerful processing capabilities within a program.


printf, sprintf

printf(“%s lines here”, $counter)

if $counter is 42, we get

42 lines here

for the output

printf(“%c lines here”, $counter)

if $counter is 42, we get

* lines here

for the output, since 42 is the ASCII value for “*”, and we’re printing a character

printf, sprintf

Some additional string formatting…

%s – output length is length($var)

%10s – output length is absolutely 10 (right justified)

%10.20s – output length is min 10, max 20

%-10.10s – output length is absolutely 10 (left justified)

Any padding is with space characters.

printf, sprintf

Some additional number formatting…

%d – output length is length($var)

%10d – output length is absolutely 10 (leading space padded)

%-10d – left justified, absolutely 10 (trailing space padded)

%-10.10d – right justified, absolutely 10 (leading zero padded)

printf, sprintf

Still more number formatting…

%f – output length is length($var)

%10.10f – guarantees 10 positions to the right of the decimal (zero padded)

printf, sprintf

printf whatever outputs to the screen

printf, sprintf


printf file whatever outputs to that file

Ex: printf file (“this is %s fun\n”, $much);

(print functions just like the above, as to output destination.)

printf, sprintf


printf file whatever outputs to that file

Ex: printf file (“this is %s fun\n”, $much);

(print functions just like the above, as to output destination.)

sprintf is just like any printf, except that its output always goes to a string variable.

Ex: $var = sprintf(“this is %s fun\n”, $much);

printf, sprintf

substr get a portion of a string

index get the location of a string in a string

length get the length of a string

ord, char convert a character to its ASCII valueand vice versa

Some other common functions

$var = ƒ(x);

uc, lc convert a string entirely to upper orlower case

ucfirst, convert the first character of a string tolcfirst upper or lower case


$var = ƒ(x);

split convert a string into pieces based on asupplied character

join convert a list of strings into one string, joinedby a supplied character


$var = ƒ(x);

@person contains a large number of people

foreach $individual (@person){ print “this is person $individual\n”;}

no subscript required!cleaner code

Loop stuffforeach, with an array


$idnum = 0;

while ($idnum < @person){ print “this is person $person[$idnum]\n”; $idnum++;}

not as clean as using foreach,but sometimes this makes more sense

Loop stuffwhile, with an array


for ($idnum=scalar(@person); $idnum--; $idnum>=0){ print “this is person $person[$idnum]\n”;}

conventional for loop

Loop stufffor, with an array

(backwards traversal)


for ($idnum=scalar(@person); $idnum--; $idnum>=0){ next if ($person[$idnum] eq “Harry”) print “this is person $person[$idnum]\n”;}

skip anybody named Harry

Loop stuff,more control


for ($idnum=scalar(@person); $idnum--; $idnum>=0){ last if ($person[$idnum] eq “Penelope”) print “this is person $person[$idnum]\n”;}next_program_line;

once we get to Penelope, leave the loop, andresume execution at next_program_line

Loop stuff,more control

@person = ();…while (“reading a file”) # this line is not real code!{ $name = substr($file_line, 0, 30); push @person, $name;}

populate an array simply,no hassle with an index variable

One last bit of array stuff…

File input and output (I/O)

“slurping” a file

File test operators

Here are a few:

-d tests if the file is a directory

-e tests if the file exists

-s returns the size of the file in bytes

-x tests if the file can be executed

Example: $filesize = -s $file

Date and Time in Perl, basic

### "create" today's date

my ($sec, $min, $hour, $day, $month, $year, $wday, $yday, $isdst) = localtime;

This gets the date and time information from the system.

### "create" today's date

my ($sec, $min, $hour, $day, $month, $year, $wday, $yday, $isdst) = localtime;

my $today = sprintf ("%4.4d.%2.2d.%2.2d", $year+1900, $month+1, $day);

This puts today’s date in “Voyager” format, 2006.04.26

Date and Time in Perl, basic

Date and Time in PerlThe program, datemath.pl, is part of your handout. The screenshot below shows its output.

Regular expressions, matching

m/PATTERN/gi

If the m for matching is not there, it is assumed.

The g modifier means to find globally, all occurrences.

The i modifier means matching case insensitive.

Modifiers are optional; others are available.

Regular expressions, substituting

s/PATTERN/REPLACEWITH/gi

The s says that substitution is the intent.

The g modifier means to substitute globally, all occurrences.

The i modifier means matching case insensitive.


Regular expressions, transliterating

tr/SEARCHFOR/REPLACEWITH/cd

The tr says that transliteration is the intent.

The c modifier means transliterate whatever is not in SEARCHFOR.

The d modifier means to delete found but unreplaced characters.


Regular expressions

# if the pattern matches

if ($var =~ /regular expression/)

{

make_something_happen;

}

Regular expressions

# if the pattern does NOT match

if ($var !~ /regular expression/)

{

make_something_happen;

}

Regular expressions

# contents of $var will be changed

# 1st occurrence of this changes to that

$var =~ s/this/that/;

# all occurrences of this are changed to that

$var =~ s/this/that/g;

Regular expressions

# contents of $var will be changed

# converts all lower case letters to# upper case letters

$var =~ tr/a-z/A-Z/;

Regular expressions

Some simple stuff to get started…

m/thisx*/ * find zero or more ‘x’ right after ‘this’

m/thisx+/ + find one or more ‘x’ right after ‘this’

m/thisx?/ ? find zero or one ‘x’ right after ‘this’

Some simple stuff to get started…

m/thisx*/ * find zero or more ‘x’ right after ‘this’

m/thisx+/ + find one or more ‘x’ right after ‘this’

m/thisx?/ ? find zero or one ‘x’ right after ‘this’

m/[0-9]{5}/ find exactly five consecutive digits

m/[0-9]{5,}/ find at least five consecutive digits

m/[0-9]{5,7}/ find from five to seven consecutive digits

Regular expressions

Some more simple stuff…

m/^this/ find ‘this’ only at the beginning of the string

m/this$/ find ‘this’ only at the end of the string

Regular expressions




Some specific characters:

\n newline (line feed)

\r carriage return

\t tab

\f form feed

\0 null

Regular expressions




Some specific characters: Some generic characters:

\n newline (line feed) \d any digit

\r carriage return \D any non-digit character

\t tab \s any whitespace character

\f form feed \S any non-whitespace character

\0 null

Regular expressions

Look in the Perl book (see Resources) for an explanation on how to use regular expressions. You can look around elsewhere, at Perl sites, and in other books, for more information and examples.

Looking at explained examples can be very helpful in learning how to use regular expressions.

(I’ve enclosed some I’ve found useful; see Resources.)

Regular expressions

Very powerful mechanism.

Often hard to understand at first glance.

Can be rather obtuse and frustrating!

If one way doesn’t work, keep at it. Most likely there is a way that works!

Regular expressions

DBI stuff

What is it and why might I want it?

DBI is the DataBase Interface module for Perl. You will also need the specific DBD (DataBase Driver) module for Oracle.

This enables Perl to perform queries against your Voyager database.

Both of these should already be on your Voyager box.

DBI stuff, how to

You need four things to connect to Voyager:

machine name your.machine.here.edu

username your_username

password your_password

SID VGER (or LIBR)

$dbh is the handle for the database

$sth is the handle for the query

Create a query…then execute it.

NOTE: SQL from Access will most likely NOT work here!

DBI stuff, how to

Get the data coming from your query.

DBI stuff, how to


You’ll need a Perl variable for each column returned in the query.

Commonly a list of variables is used; you could also use an array.

DBI stuff, how to


You’ll need a Perl variable for each column returned in the query.

Commonly a list of variables is used; you could also use an array.

Typically, you get your data in a while loop, but you could have

$var = $sth->fetchrow_array;

when you know you’re getting a single value.

DBI stuff, how to

When you’re done with a query, you should finish it. This becomes important when you have multiple queries in succession.

You can have multiple queries open at the same time. In that case, make the statement handles unique…$sth2, or $sth_patron.

Finally, you can close your database connection.

DBI stuff, how to

CPAN

Comprehensive Perl Archive Network

http://cpan.org

You name it and somebody has probably written a Perl module for it, and you’ll find it here.

There are also good Perl links here; look for the Perl Bookmarks link.

CPANInstalling modules

You need to be root for systemwide installation on Unix systems.

On Windows machines, you’ll probably need to be administrator.

You can install them “just for yourself” with a bit of tweaking, and without needing root access.

If you’re not a techie, you’ll probably want to find someone who is, to install modules.

Installing modules from CPAN is beyond the scope of this presentation.

Perl on your PC

You can get Perl for your PC from ActiveState.

They typically have two versions available; I recommend the newer one. Get the MSI version.

Installation is easy and painless, but it may take some time to complete.

A lot of modules are included with this distribution; many additional modules are available. Module installation is made easy via the Perl Package Manager (PPM).

To use ppm in ActiveState Perl, open a command prompt window and enter ppm.

Help is available by simply typing help.

Some useful commands in ppm are:

query * show what’s already installed

search pkg look for package pkg at ActiveState’s repository

install pkg retrieve and install package pkg on your machine

Perl on your PC

If you can’t find the module you’re looking for at ActiveState, you should be able to find it at CPAN, and will have to install it manually.

Perl on your PC

Voyager examples

Based on my experience (your mileage may vary), there are two main types of applications, for Voyager:


reports, or data retrievals, from the database

Voyager examples


reports, or data retrievals, from the database

data manipulation, mainly of files to be imported

Voyager examples

Voyager example,a simple report

This report finds patrons with multiple email addresses

Tells the system where to find Perl


Will be querying the Voyager database


Set up output file name


Carefully open the output file for use


Keep password data in ONE file. Why?

one point of maintenance (less work when the password changes)

reduces opportunities for error

anyone can see the source code without seeing the password data


Get some information for each patron


Get the patron identifying data in a loop, and…


Get the patron identifying data in a loop, and set up the query to get the email address(es) for this patron


In an “inner” loop, get email address data for this patron


In an “inner” loop, get email address data for this patron.

Preformat the fields for future output.


In an “inner” loop, get email address data for this patron.

Preformat the fields for future output.

Populate the address array with each address for this patron.

(note that this array starts out empty for each patron, see previous slide)


If this patron has more than one email address, then we are interested


Remove trailing spaces from the name parts, then concatenate the parts together


Now output the multiple email addresses for this patron


A sample of the output


Voyager example,some data manipulation

This program processes incoming authority records:

remove records whose 010 |a fields begin with "sj“

remaining records are stripped of the 9xx fields

Specify the file to be processed as a command line parameter.

If no parameter is supplied, display a short paragraph that shows how to use this program, then exit.


Set up the |a subfield “delimiter”.

This will be used later in the 010 field.


We could have used $ARGV[0] as the filename variable, but using $marcin makes the program more readable


An example of “slurping”, reading the file into an array without resorting to a loop


This an example of early code sticking around too long. It should be rewritten:

Insert this line before accessing the file:$/ = chr(0x1d); # use the MARC end-of-record terminator

Then get the data this way:@marcrecords = <marcin>;

The above code can be eliminated by these simple changes.



This an example of early code sticking around too long. It should be rewritten:

Insert this line before accessing the file:$/ = chr(0x1d); # use the MARC end-of-record terminator

Then get the data this way:@marcrecords = <marcin>;

The above code can be eliminated by these simple changes.

The end result is that we have an array of the MARC records from the input file

Determine the base address for data in this record, and get ready to read the directory


Get each field’s particulars, figure out where its data is, and read the data


We look for field 010, subfield a


If subfield a is found, does its data start with “sj”? If so, we do not want this record.


Looks like this record is a keeper.

If this is a 9xx field, i.e., the tag id starts with ‘9’, keep track of these fields in an array until we’ve looked at all the fields.


When done reading the record that’s a keeper, we need to delete the 9xx fields, and output the record.


If the record is not a keeper, put it in the deleted file


Resources

Learning Perl Perl in a Nutshell

Programming Perl Perl Cookbook

I use these two a lot

All books are from O’Reilly.

Advanced Perl Programming

Perl Best Practices

These will start to be useful once you have some Perl experience.

All books are from O’Reilly.

Perl Hacks

Intermediate Perl

Resources

Active State Perlhttp://activestate.com/Products/Download/Download.plex?id=ActivePerl

CPANhttp://cpan.org

a great link to links http://www.thepeoplestoolbox.com/programmers/perl

Resources

The files listed below are available at

http://homepages.wmich.edu/~zimmer/files/eugm2007

youcandoitPerl.ppt this presentation

findmanyemail.plfind patrons with multiple email addresses(available by request)

noauthsj.pldelete record if 010 |a starts with “sj”, and strip 9XX fields from remaining records

datemath.plsome program code for math with dates

snippet.grepvarious regular expressions I’ve found useful

Resources

Thanks for listening.

Questions?

[email protected]

269.387.3885

Picture © 2005 by Roy Zimmer

Thanks for listening.

Questions?

[email protected]

269.387.3885

Picture © 2006 by Roy Zimmer

you can do it! start using perl to handle your voyager needs

Technology