developer meets developer march 2011 chicago, illinois, usa roy zimmer western michigan university

Developer Meets DeveloperMarch 2011

Chicago, Illinois, USA

Roy ZimmerWestern Michigan University

Historic

• Marcive – most complex load

• MARC utilities

• Other utilities

• Patron cleanup

• Weekly index regen

Historic

Details to be found in year 2005 at http://homepages.wmich.edu/ ~zimmer/pres_index.html

Hey, it’s award winning work!

Marcive

Historic

• Marcive – most complex load

• MARC utilities

• Other utilities

• Patron cleanup

• Weekly index regen

Historic

MARC utilities

marcedit

marcread

marccount

getmarc

splitmarc

marcsearch

Find them at http://homepages.wmich.edu/~zimmer/marc_index.html

Historic

• Marcive – most complex load

• MARC utilities

• Other utilities

• Patron cleanup

• Weekly index regen

Historic

Call number sort routine(so-called for sake of brevity, really a parser and normalizer)

in a Perl program:

require cnparse.lib;

@callnumparts = cnparse($callno);

To be found at http://homepages.wmich.edu/~zimmer/other_index.html

Historic

ratiocheck.pl – database ratio re indexregen

rptarchive – cleans up /m1/voyager/$db/rpt

scheduled – what cron says should run today

To be found at http://homepages.wmich.edu/~zimmer/other_index.html

Historic

• Marcive – most complex load

• MARC utilities

• Other utilities

• Patron cleanup

• Weekly index regen

Historic

Patron cleanup

Major patron cleanup – Summer+Fall 2008

Single most current active patron barcode needed due to self-check

Details to be found in year 2009 at http://homepages.wmich.edu/~zimmer/pres_index.html

Historic

• Marcive – most complex load

• MARC utilities

• Other utilities

• Patron cleanup

• Weekly index regen

Historic

Weekly index regen

Keeps our main database in good order, due to constant loads

Yes, I check the log files!

Ongoing* – Feeds, Loads, Reports

• MeLCat

• Summon

• VuFind

• Backstage

* There are always some current changes being made to ongoing…

Ongoing – Feeds, Loads, Reports

MeLCat

daily: bib adds and updatespatron updates

monthly: bib deletes

Ongoing – Feeds, Loads, Reports

Summon

daily extract (adds and updates)

yet to come:

quarterly full extracts

daily deletions

Ongoing – Feeds, Loads, Reports

VuFind

daily: adds and updates

weekly: full extracts

Ongoing – Feeds, Loads, Reports

VuFind

Extracts on Voyager coordinated with builds on VuFind machines

Feeds go to development, near-production, and production machines

catbackup, R.I.P.

Ongoing – Feeds, Loads, Reports

Backstage

quarterly

sizeable do-not-send list

turn off all load jobs

Ongoing – Feeds, Loads, Reports

Backstage

copy of sent records is marked

load these into Voyager while Backstage processing

load returned processed records

handle Backstage reports

turn loads back on

• Yankee Book Peddler

• Emery-Pratt

• Marcive

• Alexander Street Press

Ongoing – Feeds, Loads, Reports

• All load jobs can be switched on and off

• Done via a flag file – check for presence

• Useful for Backstage processing and Voyager upgrades

• Load jobs always get all available unprocessed files, so that each run fully updates that load

Ongoing – Feeds, Loads, Reports

• Soon to come… getting load data for a particular vendor from a web page, if possible

• Plan of attack: automate with an interactive LWP session in Perl

Ongoing – Feeds, Loads, Reports

• Finish converting report generation to be entirely automated from Voyager

• Generate various Batch reports, then

• rpt2pdf.pl formats each report as needed, resulting in a .PDF file

• Print from Voyager to our “PC” printers

• Depositing files on LAN from directly from Voyager yet to come (via Samba)

Ongoing – Feeds, Loads, Reports

Recent and Current

• Implemented a local Perl library – contains a handful of common routines now called from programs – slowly starting to convert to this

• dbpctgraph.pl – generated database ratio graphs – almost ready for automatic emailing

Recent and Currentdbpctgraph output

Recent and Currentdbpctgraph output

Recent and Current

• Big Voyager upgrade in December (2010)

VoyagerOracleSolaris → to Red Hat Linuxmachinerelocations

Recent and Current

• Technical Services statistics

Recent and Current

• Technical Services statistics

• How get? Voyager queries, web scraping, VuFind queries

Recent and Current

- automating annual worksheet data retrieval

• Over 50 often multipart questions

• Mostly automating this – voyager queries, some web scraping, some remote database querying, pulling data from a LAN spreadsheet, a bit of manual input, eventually web-based interface

Picture © 2006 by Roy Zimmer

ExLibris Developer Meets Developer 2011

roy.zimmer@wmich.edu