developer meets developer march 2011 chicago, illinois, usa roy zimmer western michigan university
TRANSCRIPT
Developer Meets DeveloperMarch 2011
Chicago, Illinois, USA
Roy ZimmerWestern Michigan University
Historic
• Marcive – most complex load
• MARC utilities
• Other utilities
• Patron cleanup
• Weekly index regen
Historic
Details to be found in year 2005 at http://homepages.wmich.edu/ ~zimmer/pres_index.html
Hey, it’s award winning work!
Marcive
Historic
• Marcive – most complex load
• MARC utilities
• Other utilities
• Patron cleanup
• Weekly index regen
Historic
MARC utilities
marcedit
marcread
marccount
getmarc
splitmarc
marcsearch
Find them at http://homepages.wmich.edu/~zimmer/marc_index.html
Historic
• Marcive – most complex load
• MARC utilities
• Other utilities
• Patron cleanup
• Weekly index regen
Historic
Call number sort routine(so-called for sake of brevity, really a parser and normalizer)
in a Perl program:
require cnparse.lib;
@callnumparts = cnparse($callno);
To be found at http://homepages.wmich.edu/~zimmer/other_index.html
Historic
ratiocheck.pl – database ratio re indexregen
rptarchive – cleans up /m1/voyager/$db/rpt
scheduled – what cron says should run today
To be found at http://homepages.wmich.edu/~zimmer/other_index.html
Historic
• Marcive – most complex load
• MARC utilities
• Other utilities
• Patron cleanup
• Weekly index regen
Historic
Patron cleanup
Major patron cleanup – Summer+Fall 2008
Single most current active patron barcode needed due to self-check
Details to be found in year 2009 at http://homepages.wmich.edu/~zimmer/pres_index.html
Historic
• Marcive – most complex load
• MARC utilities
• Other utilities
• Patron cleanup
• Weekly index regen
Historic
Weekly index regen
Keeps our main database in good order, due to constant loads
Yes, I check the log files!
Ongoing* – Feeds, Loads, Reports
• MeLCat
• Summon
• VuFind
• Backstage
* There are always some current changes being made to ongoing…
Ongoing – Feeds, Loads, Reports
MeLCat
daily: bib adds and updatespatron updates
monthly: bib deletes
Ongoing – Feeds, Loads, Reports
Summon
daily extract (adds and updates)
yet to come:
quarterly full extracts
daily deletions
Ongoing – Feeds, Loads, Reports
VuFind
daily: adds and updates
weekly: full extracts
Ongoing – Feeds, Loads, Reports
VuFind
Extracts on Voyager coordinated with builds on VuFind machines
Feeds go to development, near-production, and production machines
catbackup, R.I.P.
Ongoing – Feeds, Loads, Reports
Backstage
quarterly
sizeable do-not-send list
turn off all load jobs
Ongoing – Feeds, Loads, Reports
Backstage
copy of sent records is marked
load these into Voyager while Backstage processing
load returned processed records
handle Backstage reports
turn loads back on
• Yankee Book Peddler
• Emery-Pratt
• Marcive
• Alexander Street Press
Ongoing – Feeds, Loads, Reports
• All load jobs can be switched on and off
• Done via a flag file – check for presence
• Useful for Backstage processing and Voyager upgrades
• Load jobs always get all available unprocessed files, so that each run fully updates that load
Ongoing – Feeds, Loads, Reports
• Soon to come… getting load data for a particular vendor from a web page, if possible
• Plan of attack: automate with an interactive LWP session in Perl
Ongoing – Feeds, Loads, Reports
• Finish converting report generation to be entirely automated from Voyager
• Generate various Batch reports, then
• rpt2pdf.pl formats each report as needed, resulting in a .PDF file
• Print from Voyager to our “PC” printers
• Depositing files on LAN from directly from Voyager yet to come (via Samba)
Ongoing – Feeds, Loads, Reports
Recent and Current
• Implemented a local Perl library – contains a handful of common routines now called from programs – slowly starting to convert to this
• dbpctgraph.pl – generated database ratio graphs – almost ready for automatic emailing
Recent and Currentdbpctgraph output
Recent and Currentdbpctgraph output
Recent and Current
• Big Voyager upgrade in December (2010)
VoyagerOracleSolaris → to Red Hat Linuxmachinerelocations
Recent and Current
• Technical Services statistics
Recent and Current
• Technical Services statistics
• How get? Voyager queries, web scraping, VuFind queries
Recent and Current
- automating annual worksheet data retrieval
• Over 50 often multipart questions
• Mostly automating this – voyager queries, some web scraping, some remote database querying, pulling data from a LAN spreadsheet, a bit of manual input, eventually web-based interface
