marcive documents : catching up and keeping up implementation details
Post on 20-Dec-2015
215 views
TRANSCRIPT
![Page 1: Marcive Documents : Catching Up and Keeping Up Implementation Details](https://reader035.vdocument.in/reader035/viewer/2022062714/56649d425503460f94a1d0d0/html5/thumbnails/1.jpg)
Marcive Documents:Catching Up and Keeping Up
Implementation Details
![Page 2: Marcive Documents : Catching Up and Keeping Up Implementation Details](https://reader035.vdocument.in/reader035/viewer/2022062714/56649d425503460f94a1d0d0/html5/thumbnails/2.jpg)
It took a lot of time to develop the procedure.
![Page 3: Marcive Documents : Catching Up and Keeping Up Implementation Details](https://reader035.vdocument.in/reader035/viewer/2022062714/56649d425503460f94a1d0d0/html5/thumbnails/3.jpg)
It took a lot of time to develop the procedure.
Frustration, puzzlement, head scratching…
![Page 4: Marcive Documents : Catching Up and Keeping Up Implementation Details](https://reader035.vdocument.in/reader035/viewer/2022062714/56649d425503460f94a1d0d0/html5/thumbnails/4.jpg)
It took a lot of time to develop the procedure.
Frustration, puzzlement, head scratching…
Development continued well into theEUGM submission process.
![Page 5: Marcive Documents : Catching Up and Keeping Up Implementation Details](https://reader035.vdocument.in/reader035/viewer/2022062714/56649d425503460f94a1d0d0/html5/thumbnails/5.jpg)
• Prebulk,• Bulkimport
The script makes calls to:
![Page 6: Marcive Documents : Catching Up and Keeping Up Implementation Details](https://reader035.vdocument.in/reader035/viewer/2022062714/56649d425503460f94a1d0d0/html5/thumbnails/6.jpg)
• Prebulk,• Bulkimport,• wmich8.pl, which is our implementation of the algorithm that Endeavor supplied to us for the initial remapping of Marcive data
The script makes calls to:
![Page 7: Marcive Documents : Catching Up and Keeping Up Implementation Details](https://reader035.vdocument.in/reader035/viewer/2022062714/56649d425503460f94a1d0d0/html5/thumbnails/7.jpg)
• Prebulk,• Bulkimport,• wmich8.pl, which is our implementation of the algorithm that Endeavor supplied to us for the initial remapping of Marcive data, and
• several home-grown Perl utilities
The script makes calls to:
![Page 8: Marcive Documents : Catching Up and Keeping Up Implementation Details](https://reader035.vdocument.in/reader035/viewer/2022062714/56649d425503460f94a1d0d0/html5/thumbnails/8.jpg)
Our home-grown Perl utilities:
• Marcedit inputfile outputfile .ini-file edits MARC files according to .ini directives
![Page 9: Marcive Documents : Catching Up and Keeping Up Implementation Details](https://reader035.vdocument.in/reader035/viewer/2022062714/56649d425503460f94a1d0d0/html5/thumbnails/9.jpg)
Our home-grown Perl utilities:
• Marcedit inputfile outputfile .ini-file edits MARC files according to .ini directives
• Marcread marcfile provides human-formatted output of a MARC file
![Page 10: Marcive Documents : Catching Up and Keeping Up Implementation Details](https://reader035.vdocument.in/reader035/viewer/2022062714/56649d425503460f94a1d0d0/html5/thumbnails/10.jpg)
Our home-grown Perl utilities:
• Marcedit inputfile outputfile .ini-file edits MARC files according to .ini directives
• Marcread marcfile provides human-formatted output of a MARC file
• Marccount marcfile provides a record count for the MARC file
![Page 11: Marcive Documents : Catching Up and Keeping Up Implementation Details](https://reader035.vdocument.in/reader035/viewer/2022062714/56649d425503460f94a1d0d0/html5/thumbnails/11.jpg)
Our home-grown Perl utilities:
• Marcedit is extensively used
![Page 12: Marcive Documents : Catching Up and Keeping Up Implementation Details](https://reader035.vdocument.in/reader035/viewer/2022062714/56649d425503460f94a1d0d0/html5/thumbnails/12.jpg)
Our home-grown Perl utilities:
• Marcedit is extensively used• Marcread is used in one step where a human needs to see the data
![Page 13: Marcive Documents : Catching Up and Keeping Up Implementation Details](https://reader035.vdocument.in/reader035/viewer/2022062714/56649d425503460f94a1d0d0/html5/thumbnails/13.jpg)
Our home-grown Perl utilities:
• Marcedit is extensively used• Marcread is used in one step where a human needs to see the data
• Marccount really isn’t needed, but provides useful feedback for the various steps
![Page 14: Marcive Documents : Catching Up and Keeping Up Implementation Details](https://reader035.vdocument.in/reader035/viewer/2022062714/56649d425503460f94a1d0d0/html5/thumbnails/14.jpg)
Marcive data
High-level Overview
![Page 15: Marcive Documents : Catching Up and Keeping Up Implementation Details](https://reader035.vdocument.in/reader035/viewer/2022062714/56649d425503460f94a1d0d0/html5/thumbnails/15.jpg)
wmich8.pl
Marcive data
High-level Overview
maplc.cfg
![Page 16: Marcive Documents : Catching Up and Keeping Up Implementation Details](https://reader035.vdocument.in/reader035/viewer/2022062714/56649d425503460f94a1d0d0/html5/thumbnails/16.jpg)
marcedit
wmich8.pl
Marcive data
High-level Overview
maplc.cfg
marceditxxx.ini
![Page 17: Marcive Documents : Catching Up and Keeping Up Implementation Details](https://reader035.vdocument.in/reader035/viewer/2022062714/56649d425503460f94a1d0d0/html5/thumbnails/17.jpg)
prebulk
marcedit
wmich8.pl
Marcive data
High-level Overview
maplc.cfg
marceditxxx.ini
prebulkxxx.cfg
![Page 18: Marcive Documents : Catching Up and Keeping Up Implementation Details](https://reader035.vdocument.in/reader035/viewer/2022062714/56649d425503460f94a1d0d0/html5/thumbnails/18.jpg)
Voyager
bulkimport
prebulk
marcedit
wmich8.pl
Marcive data
High-level Overview
maplc.cfg
marceditxxx.ini
prebulkxxx.cfg
![Page 19: Marcive Documents : Catching Up and Keeping Up Implementation Details](https://reader035.vdocument.in/reader035/viewer/2022062714/56649d425503460f94a1d0d0/html5/thumbnails/19.jpg)
Details...
![Page 20: Marcive Documents : Catching Up and Keeping Up Implementation Details](https://reader035.vdocument.in/reader035/viewer/2022062714/56649d425503460f94a1d0d0/html5/thumbnails/20.jpg)
Low-level Overview Step 1
marcedit
marcedit_655.ini
Marcive data
Edit 655 |2, where found
![Page 21: Marcive Documents : Catching Up and Keeping Up Implementation Details](https://reader035.vdocument.in/reader035/viewer/2022062714/56649d425503460f94a1d0d0/html5/thumbnails/21.jpg)
Low-level Overview Step 1
Edit 655 |2, where found
marcedit_655.ini
[ADD]
[REMOVE]
[EDIT]
replacesubfield|655|2|local|LCSH
[FIND]
![Page 22: Marcive Documents : Catching Up and Keeping Up Implementation Details](https://reader035.vdocument.in/reader035/viewer/2022062714/56649d425503460f94a1d0d0/html5/thumbnails/22.jpg)
Low-level Overview Step 2
wmich8.pl
maplc.cfg
notremapped
prebulkremapped
prebulk_lc_yes.cfg
to bulkimport
Remap GPO number to LC number where possible
![Page 23: Marcive Documents : Catching Up and Keeping Up Implementation Details](https://reader035.vdocument.in/reader035/viewer/2022062714/56649d425503460f94a1d0d0/html5/thumbnails/23.jpg)
Low-level Overview Step 2
Remap GPO number to LC number where possible
map item?
wmich8.pl detail
not remapped
no
![Page 24: Marcive Documents : Catching Up and Keeping Up Implementation Details](https://reader035.vdocument.in/reader035/viewer/2022062714/56649d425503460f94a1d0d0/html5/thumbnails/24.jpg)
Low-level Overview Step 2
Remap GPO number to LC number where possible
map item? has 074 |a?
wmich8.pl detail
not remapped
no no
yes
![Page 25: Marcive Documents : Catching Up and Keeping Up Implementation Details](https://reader035.vdocument.in/reader035/viewer/2022062714/56649d425503460f94a1d0d0/html5/thumbnails/25.jpg)
Low-level Overview Step 2
Remap GPO number to LC number where possible
map item? has 074 |a? GPO # in table?
wmich8.pl detail
not remapped
no no no
yes yes
![Page 26: Marcive Documents : Catching Up and Keeping Up Implementation Details](https://reader035.vdocument.in/reader035/viewer/2022062714/56649d425503460f94a1d0d0/html5/thumbnails/26.jpg)
Low-level Overview Step 2
Remap GPO number to LC number where possible
map item? has 074 |a? GPO # in table?
099 found?
delete 099
add 099 with LC # from table
wmich8.pl detail
not remapped remapped
no no no
yes yes yes
yes
no
![Page 27: Marcive Documents : Catching Up and Keeping Up Implementation Details](https://reader035.vdocument.in/reader035/viewer/2022062714/56649d425503460f94a1d0d0/html5/thumbnails/27.jpg)
Low-level Overview Step 2
maplc.cfg
619-M-42 G 3961s .C1 24-250 .G35
0619-P-17 G 3703s .C1 50-100 .U57
0378-E-05 G 3201s .P6 TPC- .G73
0191-B-14 G 3311s .P5 5-2,100 .N25
Remap GPO number to LC number where possible
![Page 28: Marcive Documents : Catching Up and Keeping Up Implementation Details](https://reader035.vdocument.in/reader035/viewer/2022062714/56649d425503460f94a1d0d0/html5/thumbnails/28.jpg)
prebulk_lc_yes.cfg
![Page 29: Marcive Documents : Catching Up and Keeping Up Implementation Details](https://reader035.vdocument.in/reader035/viewer/2022062714/56649d425503460f94a1d0d0/html5/thumbnails/29.jpg)
prebulk_lc_yes.cfg
Low-level Overview Step 2
![Page 30: Marcive Documents : Catching Up and Keeping Up Implementation Details](https://reader035.vdocument.in/reader035/viewer/2022062714/56649d425503460f94a1d0d0/html5/thumbnails/30.jpg)
prebulk_lc_yes.cfg
Low-level Overview Step 2
![Page 31: Marcive Documents : Catching Up and Keeping Up Implementation Details](https://reader035.vdocument.in/reader035/viewer/2022062714/56649d425503460f94a1d0d0/html5/thumbnails/31.jpg)
Low-level Overview Step 3
marcedit
marcedit_yesmap.ini
prebulk
prebulk_lc_no.cfg
to bulkimport
notremapped
marcread
email to recipients
Extract a copy of records with 049 |a wmap,map
copy extract
![Page 32: Marcive Documents : Catching Up and Keeping Up Implementation Details](https://reader035.vdocument.in/reader035/viewer/2022062714/56649d425503460f94a1d0d0/html5/thumbnails/32.jpg)
Low-level Overview Step 3
marcedit_yesmap.ini
[ADD]
[REMOVE]
[EDIT]
[FIND]
049||||||a||wmap,map
Extract a copy of records with 049 |a wmap,map
![Page 33: Marcive Documents : Catching Up and Keeping Up Implementation Details](https://reader035.vdocument.in/reader035/viewer/2022062714/56649d425503460f94a1d0d0/html5/thumbnails/33.jpg)
Low-level Overview Step 3
prebulk_lc_no.cfg (differences)
Extract a copy of records with 049 |a wmap,map
![Page 34: Marcive Documents : Catching Up and Keeping Up Implementation Details](https://reader035.vdocument.in/reader035/viewer/2022062714/56649d425503460f94a1d0d0/html5/thumbnails/34.jpg)
Low-level Overview Step 3
Extract a copy of records with 049 |a wmap,map
Extracts bib IDs from a log.imp* file to another file. It optionally precedes each number with an R or and an A, to indicate if that record was replaced or added.
Example:
Useful at this stage...
implogidextract.pl log.impxxx outfile desc
![Page 35: Marcive Documents : Catching Up and Keeping Up Implementation Details](https://reader035.vdocument.in/reader035/viewer/2022062714/56649d425503460f94a1d0d0/html5/thumbnails/35.jpg)
Low-level Overview Step 4
marcedit
marcedit_yesnaut.ini
prebulk
prebulk_lc_no.cfg
to bulk import
Extract a copy of records with 049 |a wmap,naut
copy extract
![Page 36: Marcive Documents : Catching Up and Keeping Up Implementation Details](https://reader035.vdocument.in/reader035/viewer/2022062714/56649d425503460f94a1d0d0/html5/thumbnails/36.jpg)
Low-level Overview Step 4
Extract a copy of records with 049 |a wmap,naut
marcedit_yesnaut.ini
[ADD]
[REMOVE]
[EDIT]
[FIND]
049||||||a||wmap,naut
![Page 37: Marcive Documents : Catching Up and Keeping Up Implementation Details](https://reader035.vdocument.in/reader035/viewer/2022062714/56649d425503460f94a1d0d0/html5/thumbnails/37.jpg)
Low-level Overview Step 4
prebulk_lc_no.cfg
Extract a copy of records with 049 |a wmap,naut
![Page 38: Marcive Documents : Catching Up and Keeping Up Implementation Details](https://reader035.vdocument.in/reader035/viewer/2022062714/56649d425503460f94a1d0d0/html5/thumbnails/38.jpg)
Low-level Overview Step 5
marcedit
marcedit_www_yes_minus_099.ini
Remove the 099 field from records containing 049 |a wdoc,www
![Page 39: Marcive Documents : Catching Up and Keeping Up Implementation Details](https://reader035.vdocument.in/reader035/viewer/2022062714/56649d425503460f94a1d0d0/html5/thumbnails/39.jpg)
Low-level Overview Step 5
Remove the 099 field from records containing 049 |a wdoc,www
marcedit_www_yes_minus_099.ini
[ADD]
[REMOVE]
099
[EDIT]
[FIND]
049||||||a||wdoc,www
![Page 40: Marcive Documents : Catching Up and Keeping Up Implementation Details](https://reader035.vdocument.in/reader035/viewer/2022062714/56649d425503460f94a1d0d0/html5/thumbnails/40.jpg)
Low-level Overview Step 6
marcedit
marcedit_www_yes_add_099.ini
prebulk
prebulk_www_yes.cfg
to bulkimport
Add our own 099 field to records containing 049 |a wdoc,www and extract a copy
copy extract
![Page 41: Marcive Documents : Catching Up and Keeping Up Implementation Details](https://reader035.vdocument.in/reader035/viewer/2022062714/56649d425503460f94a1d0d0/html5/thumbnails/41.jpg)
Low-level Overview Step 6
Add our own 099 field to records containing 049 |a wdoc,www and extract a copy
marcedit_www_yes_add_099.ini
[ADD]
099||9|1|a|_
[REMOVE]
[EDIT]
[FIND]
049||||||a||wdoc,www
![Page 42: Marcive Documents : Catching Up and Keeping Up Implementation Details](https://reader035.vdocument.in/reader035/viewer/2022062714/56649d425503460f94a1d0d0/html5/thumbnails/42.jpg)
Low-level Overview Step 6
Add our own 099 field to records containing 049 |a wdoc,www and extract a copy
prebulk_www_yes.cfg (differences)
![Page 43: Marcive Documents : Catching Up and Keeping Up Implementation Details](https://reader035.vdocument.in/reader035/viewer/2022062714/56649d425503460f94a1d0d0/html5/thumbnails/43.jpg)
Low-level Overview Step 7
marcedit
marcedit_nomap.ini
Remove records containing 049 |a wmap,map
![Page 44: Marcive Documents : Catching Up and Keeping Up Implementation Details](https://reader035.vdocument.in/reader035/viewer/2022062714/56649d425503460f94a1d0d0/html5/thumbnails/44.jpg)
Low-level Overview Step 7
Remove records containing 049 |a wmap,map
marcedit_nomap.ini
[ADD]
[REMOVE]
Record
[EDIT]
[FIND]
not
049||||||a||wmap,map
![Page 45: Marcive Documents : Catching Up and Keeping Up Implementation Details](https://reader035.vdocument.in/reader035/viewer/2022062714/56649d425503460f94a1d0d0/html5/thumbnails/45.jpg)
Low-level Overview Step 8
marcedit
marcedit_nonaut.ini
Remove records containing 049 |a wmap,naut
![Page 46: Marcive Documents : Catching Up and Keeping Up Implementation Details](https://reader035.vdocument.in/reader035/viewer/2022062714/56649d425503460f94a1d0d0/html5/thumbnails/46.jpg)
Low-level Overview Step 8
Remove records containing 049 |a wmap,naut
marcedit_nonaut.ini
[ADD]
[REMOVE]
Record
[EDIT]
[FIND]
not
049||||||a||wmap,naut
![Page 47: Marcive Documents : Catching Up and Keeping Up Implementation Details](https://reader035.vdocument.in/reader035/viewer/2022062714/56649d425503460f94a1d0d0/html5/thumbnails/47.jpg)
Low-level Overview Step 9
marcedit
marcedit_nowww.ini
Remove records containing 049 |a wdoc,www
![Page 48: Marcive Documents : Catching Up and Keeping Up Implementation Details](https://reader035.vdocument.in/reader035/viewer/2022062714/56649d425503460f94a1d0d0/html5/thumbnails/48.jpg)
Low-level Overview Step 9
Remove records containing 049 |a wdoc,www
marcedit_nowww.ini
[ADD]
[REMOVE]
Record
[EDIT]
[FIND]
not
049||||||a||wdoc,www
![Page 49: Marcive Documents : Catching Up and Keeping Up Implementation Details](https://reader035.vdocument.in/reader035/viewer/2022062714/56649d425503460f94a1d0d0/html5/thumbnails/49.jpg)
Low-level Overview Step 10
prebulk
prebulk_www_no.cfg
Run what’s left through prebulk
![Page 50: Marcive Documents : Catching Up and Keeping Up Implementation Details](https://reader035.vdocument.in/reader035/viewer/2022062714/56649d425503460f94a1d0d0/html5/thumbnails/50.jpg)
Low-level Overview Step 10
Run what’s left through prebulk
prebulk_www_no.cfg (differences)
![Page 51: Marcive Documents : Catching Up and Keeping Up Implementation Details](https://reader035.vdocument.in/reader035/viewer/2022062714/56649d425503460f94a1d0d0/html5/thumbnails/51.jpg)
Low-level Overview Step 11
bulkimport
Marcive bulk import rules
Run everything through bulkimport
Voyager
Records that were previously copied/extracted and run through
prebulk
![Page 52: Marcive Documents : Catching Up and Keeping Up Implementation Details](https://reader035.vdocument.in/reader035/viewer/2022062714/56649d425503460f94a1d0d0/html5/thumbnails/52.jpg)
Resources
implogidextract.pl
http://homepages.wmich.edu/~zimmer/marc_index.htmlmarcedit.pl
marcread.pl
marccount.pl
http://homepages.wmich.edu/~zimmer/other_index.html