a strand of perls:

A Strand of Perls:

SomeHome Grown

Utilities

Our New Books List

Call Number Sorting

Getting Operator Profiles

QPID – Quick Patron

Information Dump (cupid…)

Syllabus

Our New Books List

Call Number Sorting




Why present another new books list?Different strokes for different folks…

Given:

professors don’t care about call numbers; they just want to go to their area and see what’s new – information sorted by department


Given:


information needed on a monthly basis


Given:



can go back through data for several previous months


Given:



can go back through data for several previous months

here’s an overview…

New Books List Process at WMU

get last month’s

acquisitions


get last month’s

acquisitions

break up by department


get last month’s

acquisitions

Department X

Department C

Department B

Department A



get last month’s

acquisitions

Department X

Department C

Department B

Department A


one text output file


ftp to Batch PC

get last month’s

acquisitions

Department X

Department C

Department B

Department A




put on library LAN for the Web Office

ftp to Batch PC

get last month’s

acquisitions

Department X

Department C

Department B

Department A




Our production-type jobs get the database password from a file, for easy maintenance.

Then use DBI to set up access to the database.


The query (sprintf wrapper removed for clarity)


Get data from the query in a loop and put in an array


Get rid of headphones!


Create sort vector and put in array


Got the deduping code from one of the O’Reilly Perl books.

Data will implicitly be in call number order due to sort vector structure.

…line noise…?

Digression…

Speaking of line noise…

Broken up for clarity

This puts the line count of a MARC file into a shell script.


Now we need to get the results classified by department, going by call number ranges.

raw ranges file…


The call number range specifications are normalized in the same manner used for sorting.


The call number range specifications are normalized in the same manner used for sorting.

Great for the computer, not so easy for us humans.

Created a utility to make a human-readable version.

formatted ranges file…


The range data is read into arrays. (If a syntactic error was found, the program stops and shows where it is.)


The range data is read into arrays. (If a syntactic error was found, the program stops and shows where it is.)

Then loop for each department. If the current call number falls within the current range, it goes into the current department file.


The output files are sorted.

For our final processing, we loop through each of these sorted files of raw data. We ignore the call number chunks created during the normalization process.

The desired fields are concatenated and line-wrapped.

field1 | field2 | etc.

how this was done…


As we loop through the contents of each departmental file:

We split up the sort vector, and store the output fields with a vertical bar in between.


Some additional processing is done, including the always visually entertaining regular expression manipulations.


The output is line-wrapped prior to writing to the file.


The output is line-wrapped prior to writing to the file.

You’ll need some initial setup for the above wrap to work.


Now we have our output file…

When we first implemented our list, this was the whole process. The file was handed off to the library, where staff separated the departmental data out of the file, added the HTML, and put it on our web site.

It took several hours to do this!


Once I knew this, I looked into further automation. Now we have an additional Perl script that takes care of the rest of the story.

I looked at the new books’ web pages the library had created and figured out that I could break out three sections of static html.


We read the previous output file, paying attention to which department we’re “in”.



Next, we create a separate .html file for each department, incorporating the static HTML sections, adding date information where necessary.



Next, we create a separate .html file for each department, incorporating the static HTML sections, adding date information where necessary.

Finally, these files are put on the library LAN and a reminder email is sent out.


put on library LAN for the Web Office

ftp to Batch PC

get last month’s

acquisitions

Department X

Department C

Department B

Department A

static HTML

static HTML

static HTML

static HTML


separate HTML file for each department, ready to be

incorporated into the library web pages


See the results at:

http://www.wmich.edu/library/newbooks/index.html

Our New Books List

Call Number Sorting




Call Number Sorting

Seems right to call it sorting, but it’s really in the normalization process that the “magic” occurs.

Call Number Sorting


Uses intelligent parsing, not a quick regular expression implementation.

Call Number Sorting



Designed with LC call numbers in mind, but pretty much handles everything, including locally generated call numbers.

Call Number Sorting




Resulting sorts appear to be about 99% accurate (my estimate).

Call Number Sorting




Resulting sorts appear to be about 99% accurate (my estimate).

The algorithm divides call numbers into chunks, based on separators.

Call Number Sorting

Explicit separators: colon (:)

Call Number Sorting

Explicit separators: colon (:)semicolon (;)

Call Number Sorting

Explicit separators: colon (:)semicolon (;)comma (,)

Call Number Sorting

Explicit separators: colon (:)semicolon (;)comma (,)period (.)

Call Number Sorting

Explicit separators: colon (:)semicolon (;)comma (,)period (.)space ( )

Call Number Sorting

Explicit separators: colon (:)semicolon (;)comma (,)period (.)space ( )forward slash (/)

Call Number Sorting


Implicit separators:transitions: alpha->numeric

numeric->alpha

Call Number Sorting


Implicit separators:transitions: alpha->numeric

numeric->alpha

During parsing, separators are absorbed, but the period may be uniquely retained.

Call Number Sorting

Further processing and normalization include:

Whole numbers are treated differently from decimal numbers.

Call Number Sorting



Decimal numbers may affect as many as several following chunks.

Call Number Sorting



Decimal numbers may affect as many as several following chunks.

Look-ahead and look-back for one or more chunks is also employed.

Call Number Sorting

demo… democall.pl democall.lst

Call Number Sorting

This code is available at:

http://homepages.wmich.edu/~zimmer

Our New Books List

Call Number Sorting




The next two programs are designed to run on PCs, not on a Voyager box.

In order to run them, you will need:

Perl, (I use ActiveState) DBI and DBD for Oracle (get from ActiveState) Oracle Client software (from Oracle)

preface

Get ActiveState Perl at:http://www.activestate.com/Products/Download/Register.plex?id=ActivePerl

This puts you at the registration (optional) screen for the download. At the next page, you’ll probably want to select the “MSI” installation for Windows.

Get version 5.6.1.

preface

How to get DBI and DBD:Once ActivePerl is installed, open a command prompt window (DOS prompt)

Run PPM

Once in PPM, install DBI and DBD

Exit PPM

preface

Oracle Client softwareRequired! DBI and DBD rely on this.

Check to make sure that the Oracle licensing arrangement at your site allows you to install the client software, if you do not already have a suitably equipped PC available.

I used 8.1.6.

The stated combination of versions is the only one I got to work. This is on machines running Windows 2000.

preface


Demo getprof…


Program outline:

Setup and initialization

Look for each possible profile; when found:get count of affected locationsget profile info

format Y/N boolean values output info to file in HTML format

Invoke browser to display profile data


Query example using the master profile


Do some data massaging,

then use tables in HTML for formatting.

Our New Books List

Call Number Sorting




Patron Information Dump

Demo patdump…

Patron Information Dump

Program outline:

Setup and initialization.

Run a series of queries to get all the patron data.

Take resulting data from each query and format in html, creating a file.

Invoke browser to display patron data.

Both the operator profile and patron dump programs allow for choice of browser, via an .ini file, so that different users can use different browsers.

Sample .ini file

Some users like to explore…

Slow down or stop them by associating an icon with the .bat file and hiding the other files.

.batnormal file

your choice of browser

.inihidden, read-only

.plhidden, read-only

.htmlnormal file

The code for these two programs is available at:


Resources

http://www.tek-tips.com/gfaqs.cfm/pid/219/fid/1711

for general installation issues on a PC

http://metalink.oracle.com

login, then search for 131299.1 for Pentium IV problems with the client software install

http://www.activestate.com/Products/Download/Register.plex?id=ActivePerl

get your ActivePerl here


Western Michigan University new books list


some of the code from this presentation

http://www.tek-tips.com/gfaqs.cfm/pid/219/fid/1711

http://metalink.oracle.com/






Questions?Email: [email protected]: 269.387.3885

Thanks for listening.

a strand of perls:

Documents

arraynew books list

claritynew books list

overviewnew books list

whats new information

oreilly perl books

department information

different strokes

different folksgiven