catmandu - swibswib.org/swib13/slides/jahn_swib13_128.pdf · what is it? • a perl library • a...
TRANSCRIPT
![Page 1: Catmandu - SWIBswib.org/swib13/slides/jahn_swib13_128.pdf · What is it? • a Perl library • a command line tool • to import, transform and export (library) data • in a pragmatic](https://reader031.vdocument.in/reader031/viewer/2022022006/5ac46ca07f8b9a220b8cd175/html5/thumbnails/1.jpg)
Catmandu
![Page 2: Catmandu - SWIBswib.org/swib13/slides/jahn_swib13_128.pdf · What is it? • a Perl library • a command line tool • to import, transform and export (library) data • in a pragmatic](https://reader031.vdocument.in/reader031/viewer/2022022006/5ac46ca07f8b9a220b8cd175/html5/thumbnails/2.jpg)
What is it?
• a Perl library
• a command line tool
• to import, transform and export (library) data
• in a pragmatic way
• can handle large streams of data
![Page 3: Catmandu - SWIBswib.org/swib13/slides/jahn_swib13_128.pdf · What is it? • a Perl library • a command line tool • to import, transform and export (library) data • in a pragmatic](https://reader031.vdocument.in/reader031/viewer/2022022006/5ac46ca07f8b9a220b8cd175/html5/thumbnails/3.jpg)
Where do i find it?
• http://librecat.org/
• https://github.com/LibreCat
• http://search.cpan.org/search?query=Catmandu
![Page 4: Catmandu - SWIBswib.org/swib13/slides/jahn_swib13_128.pdf · What is it? • a Perl library • a command line tool • to import, transform and export (library) data • in a pragmatic](https://reader031.vdocument.in/reader031/viewer/2022022006/5ac46ca07f8b9a220b8cd175/html5/thumbnails/4.jpg)
Show of hands
• programming?
• json?
• command line user?
![Page 5: Catmandu - SWIBswib.org/swib13/slides/jahn_swib13_128.pdf · What is it? • a Perl library • a command line tool • to import, transform and export (library) data • in a pragmatic](https://reader031.vdocument.in/reader031/viewer/2022022006/5ac46ca07f8b9a220b8cd175/html5/thumbnails/5.jpg)
Show me$ catmandu convert JSON to YAML
!
$ catmandu convert JSON
--file /path/to/file.yaml
to YAML
--file /path/to/file.json
--fix 'capitalize("title")'
--fix 'trim("abstract")'
![Page 6: Catmandu - SWIBswib.org/swib13/slides/jahn_swib13_128.pdf · What is it? • a Perl library • a command line tool • to import, transform and export (library) data • in a pragmatic](https://reader031.vdocument.in/reader031/viewer/2022022006/5ac46ca07f8b9a220b8cd175/html5/thumbnails/6.jpg)
Show me$ catmandu import MARC
--file /path/to/records.xml
--type MARCXML
to MongoDB
--database-name catalogue
--bag records
--verbose
![Page 7: Catmandu - SWIBswib.org/swib13/slides/jahn_swib13_128.pdf · What is it? • a Perl library • a command line tool • to import, transform and export (library) data • in a pragmatic](https://reader031.vdocument.in/reader031/viewer/2022022006/5ac46ca07f8b9a220b8cd175/html5/thumbnails/7.jpg)
Show me$ catmandu import MARC
--file /path/to/records.xml
--type MARCXML
to MongoDB
--database-name catalogue
--bag records
--verbose
--fix "marc_map('245','title')"
--fix "marc_map('100','authors.\$append')"
--fix "marc_map('008/35-35','language')"
![Page 8: Catmandu - SWIBswib.org/swib13/slides/jahn_swib13_128.pdf · What is it? • a Perl library • a command line tool • to import, transform and export (library) data • in a pragmatic](https://reader031.vdocument.in/reader031/viewer/2022022006/5ac46ca07f8b9a220b8cd175/html5/thumbnails/8.jpg)
Commands$ catmandu convert
convert data from one file format into another!
!
$ catmandu import
import data from a file into a store!
!
$ catmandu export
export data from a store into a file!
!
$ catmandu move
copy data from a store into another store!
!
$ catmandu count
count the number of objects in a store!
!
$ catmandu delete
delete objects from a store
![Page 9: Catmandu - SWIBswib.org/swib13/slides/jahn_swib13_128.pdf · What is it? • a Perl library • a command line tool • to import, transform and export (library) data • in a pragmatic](https://reader031.vdocument.in/reader031/viewer/2022022006/5ac46ca07f8b9a220b8cd175/html5/thumbnails/9.jpg)
Commands
$ catmandu repl
![Page 10: Catmandu - SWIBswib.org/swib13/slides/jahn_swib13_128.pdf · What is it? • a Perl library • a command line tool • to import, transform and export (library) data • in a pragmatic](https://reader031.vdocument.in/reader031/viewer/2022022006/5ac46ca07f8b9a220b8cd175/html5/thumbnails/10.jpg)
In Perluse Catmandu;
!
my $importer = Catmandu->importer('CSV',
fields => ['person_id', 'name']);
!
my $bag = Catmandu->store('ElasticSearch',
index_name => "myapp")->bag("people");
!
my $exporter = Catmandu->exporter('JSON', file => $out);
!
$bag->add_many($importer);
$bag->add({person_id => "123", name => "mr. jones"});
$bag->commit;
!
$exporter->add_many($bag);
![Page 11: Catmandu - SWIBswib.org/swib13/slides/jahn_swib13_128.pdf · What is it? • a Perl library • a command line tool • to import, transform and export (library) data • in a pragmatic](https://reader031.vdocument.in/reader031/viewer/2022022006/5ac46ca07f8b9a220b8cd175/html5/thumbnails/11.jpg)
In Perluse Catmandu;
!
my $importer = Catmandu->importer('CSV',
fields => ['person_id', 'name']);
!
my $fixer = Catmandu->fixer([
'/path/to/fix/file.txt',
'capitalize("name")',
]);
!
$importer = $fixer->fix($importer);
!
$importer->each(sub {
my $person = shift;
say $person->{"name"};
});
![Page 12: Catmandu - SWIBswib.org/swib13/slides/jahn_swib13_128.pdf · What is it? • a Perl library • a command line tool • to import, transform and export (library) data • in a pragmatic](https://reader031.vdocument.in/reader031/viewer/2022022006/5ac46ca07f8b9a220b8cd175/html5/thumbnails/12.jpg)
Fix file example
add_field('my.deeply.nested.field', "value");
add_field('my.list.$append', "value");
!
remove_field('my.list.3');
remove_field('my.list.$last');
!
if_exists('my.key');
cmd('python transform.py');
end();
![Page 13: Catmandu - SWIBswib.org/swib13/slides/jahn_swib13_128.pdf · What is it? • a Perl library • a command line tool • to import, transform and export (library) data • in a pragmatic](https://reader031.vdocument.in/reader031/viewer/2022022006/5ac46ca07f8b9a220b8cd175/html5/thumbnails/13.jpg)
Internal data model
• plain data, no objects
• basically everything that is representable as JSON
{title => "my title",
authors => [
{name => "mr. jones"},
{name => "mr. smith"}],
weight => 1.73,
}
![Page 14: Catmandu - SWIBswib.org/swib13/slides/jahn_swib13_128.pdf · What is it? • a Perl library • a command line tool • to import, transform and export (library) data • in a pragmatic](https://reader031.vdocument.in/reader031/viewer/2022022006/5ac46ca07f8b9a220b8cd175/html5/thumbnails/14.jpg)
Main Catmandu parts• Catmandu
• Catmandu::Importer (Iterable)
• Catmandu::Exporter (Addable, Fixable)
• Catmandu::Store (Addable, Fixable, Iterable)
• Catmandu::Bag (Addable, Fixable, Iterable[, Searchable])
• Catmandu::Hits (Iterable)
• Catmandu::Fix Catmandu::Fix::Base Catmandu::Fix::Condition
![Page 15: Catmandu - SWIBswib.org/swib13/slides/jahn_swib13_128.pdf · What is it? • a Perl library • a command line tool • to import, transform and export (library) data • in a pragmatic](https://reader031.vdocument.in/reader031/viewer/2022022006/5ac46ca07f8b9a220b8cd175/html5/thumbnails/15.jpg)
Importers• Atom
• CSV
• JSON
• YAML
• MARC
• MAB
• ArXiv
• CrossRef
• LDAP
• OAI
• PLoS
• PubMed
• SRU
• ORCID
• Z39.50
• Inspire
![Page 16: Catmandu - SWIBswib.org/swib13/slides/jahn_swib13_128.pdf · What is it? • a Perl library • a command line tool • to import, transform and export (library) data • in a pragmatic](https://reader031.vdocument.in/reader031/viewer/2022022006/5ac46ca07f8b9a220b8cd175/html5/thumbnails/16.jpg)
Importers• MediaMosa
• AlephX
![Page 17: Catmandu - SWIBswib.org/swib13/slides/jahn_swib13_128.pdf · What is it? • a Perl library • a command line tool • to import, transform and export (library) data • in a pragmatic](https://reader031.vdocument.in/reader031/viewer/2022022006/5ac46ca07f8b9a220b8cd175/html5/thumbnails/17.jpg)
Stores• DBI
• MongoDB
• ElasticSearch
• Solr
• FedoraCommons
• CouchDB
• Hash
![Page 18: Catmandu - SWIBswib.org/swib13/slides/jahn_swib13_128.pdf · What is it? • a Perl library • a command line tool • to import, transform and export (library) data • in a pragmatic](https://reader031.vdocument.in/reader031/viewer/2022022006/5ac46ca07f8b9a220b8cd175/html5/thumbnails/18.jpg)
Exporters• Atom
• BibTeX
• CSV
• JSON
• RIS
• Template
• XLS
• YAML
• MARCXML
• RTF
• ODS
![Page 19: Catmandu - SWIBswib.org/swib13/slides/jahn_swib13_128.pdf · What is it? • a Perl library • a command line tool • to import, transform and export (library) data • in a pragmatic](https://reader031.vdocument.in/reader031/viewer/2022022006/5ac46ca07f8b9a220b8cd175/html5/thumbnails/19.jpg)
Fixes
• add_field
• append
• capitalize
• clone
• collapse
• copy_field
• downcase
• expand
• join_field
• move_field
• nothing
• prepend
• remove_field
![Page 20: Catmandu - SWIBswib.org/swib13/slides/jahn_swib13_128.pdf · What is it? • a Perl library • a command line tool • to import, transform and export (library) data • in a pragmatic](https://reader031.vdocument.in/reader031/viewer/2022022006/5ac46ca07f8b9a220b8cd175/html5/thumbnails/20.jpg)
Fixes
• replace_all
• retain_field
• set_field
• split_field
• substring
• trim
• upcase
• marc_map
• marc_in_json
• marc_xml
• mab_map
• mab_in_json
• mab_xml
• cmd
![Page 21: Catmandu - SWIBswib.org/swib13/slides/jahn_swib13_128.pdf · What is it? • a Perl library • a command line tool • to import, transform and export (library) data • in a pragmatic](https://reader031.vdocument.in/reader031/viewer/2022022006/5ac46ca07f8b9a220b8cd175/html5/thumbnails/21.jpg)
Fixes
• sum
• lookup
• lookup_in_store
• to_json
• from_json
![Page 22: Catmandu - SWIBswib.org/swib13/slides/jahn_swib13_128.pdf · What is it? • a Perl library • a command line tool • to import, transform and export (library) data • in a pragmatic](https://reader031.vdocument.in/reader031/viewer/2022022006/5ac46ca07f8b9a220b8cd175/html5/thumbnails/22.jpg)
Fixes (conditionals)
• if_all_match
• unless_all_match
• if_any_match
• unless_any_match
• if_exists
• unless_exists
• otherwise
• end
![Page 23: Catmandu - SWIBswib.org/swib13/slides/jahn_swib13_128.pdf · What is it? • a Perl library • a command line tool • to import, transform and export (library) data • in a pragmatic](https://reader031.vdocument.in/reader031/viewer/2022022006/5ac46ca07f8b9a220b8cd175/html5/thumbnails/23.jpg)
RDF in Catmandu
Monday 2 December 13
![Page 24: Catmandu - SWIBswib.org/swib13/slides/jahn_swib13_128.pdf · What is it? • a Perl library • a command line tool • to import, transform and export (library) data • in a pragmatic](https://reader031.vdocument.in/reader031/viewer/2022022006/5ac46ca07f8b9a220b8cd175/html5/thumbnails/24.jpg)
Monday 2 December 13
![Page 25: Catmandu - SWIBswib.org/swib13/slides/jahn_swib13_128.pdf · What is it? • a Perl library • a command line tool • to import, transform and export (library) data • in a pragmatic](https://reader031.vdocument.in/reader031/viewer/2022022006/5ac46ca07f8b9a220b8cd175/html5/thumbnails/25.jpg)
MongoAdmin
Monday 2 December 13
![Page 26: Catmandu - SWIBswib.org/swib13/slides/jahn_swib13_128.pdf · What is it? • a Perl library • a command line tool • to import, transform and export (library) data • in a pragmatic](https://reader031.vdocument.in/reader031/viewer/2022022006/5ac46ca07f8b9a220b8cd175/html5/thumbnails/26.jpg)
http://ec2-50-17-116-137.compute-1.amazonaws.com
swib2013/swib2013
Monday 2 December 13
![Page 27: Catmandu - SWIBswib.org/swib13/slides/jahn_swib13_128.pdf · What is it? • a Perl library • a command line tool • to import, transform and export (library) data • in a pragmatic](https://reader031.vdocument.in/reader031/viewer/2022022006/5ac46ca07f8b9a220b8cd175/html5/thumbnails/27.jpg)
Monday 2 December 13
![Page 28: Catmandu - SWIBswib.org/swib13/slides/jahn_swib13_128.pdf · What is it? • a Perl library • a command line tool • to import, transform and export (library) data • in a pragmatic](https://reader031.vdocument.in/reader031/viewer/2022022006/5ac46ca07f8b9a220b8cd175/html5/thumbnails/28.jpg)
Monday 2 December 13
![Page 29: Catmandu - SWIBswib.org/swib13/slides/jahn_swib13_128.pdf · What is it? • a Perl library • a command line tool • to import, transform and export (library) data • in a pragmatic](https://reader031.vdocument.in/reader031/viewer/2022022006/5ac46ca07f8b9a220b8cd175/html5/thumbnails/29.jpg)
Monday 2 December 13
![Page 30: Catmandu - SWIBswib.org/swib13/slides/jahn_swib13_128.pdf · What is it? • a Perl library • a command line tool • to import, transform and export (library) data • in a pragmatic](https://reader031.vdocument.in/reader031/viewer/2022022006/5ac46ca07f8b9a220b8cd175/html5/thumbnails/30.jpg)
Monday 2 December 13
![Page 31: Catmandu - SWIBswib.org/swib13/slides/jahn_swib13_128.pdf · What is it? • a Perl library • a command line tool • to import, transform and export (library) data • in a pragmatic](https://reader031.vdocument.in/reader031/viewer/2022022006/5ac46ca07f8b9a220b8cd175/html5/thumbnails/31.jpg)
Monday 2 December 13
![Page 32: Catmandu - SWIBswib.org/swib13/slides/jahn_swib13_128.pdf · What is it? • a Perl library • a command line tool • to import, transform and export (library) data • in a pragmatic](https://reader031.vdocument.in/reader031/viewer/2022022006/5ac46ca07f8b9a220b8cd175/html5/thumbnails/32.jpg)
Monday 2 December 13
![Page 33: Catmandu - SWIBswib.org/swib13/slides/jahn_swib13_128.pdf · What is it? • a Perl library • a command line tool • to import, transform and export (library) data • in a pragmatic](https://reader031.vdocument.in/reader031/viewer/2022022006/5ac46ca07f8b9a220b8cd175/html5/thumbnails/33.jpg)
NotePad (Windows) | TextEdit (Mac) | Vi (Linux) | http://www.editpad.org/ (Online)
Monday 2 December 13
![Page 34: Catmandu - SWIBswib.org/swib13/slides/jahn_swib13_128.pdf · What is it? • a Perl library • a command line tool • to import, transform and export (library) data • in a pragmatic](https://reader031.vdocument.in/reader031/viewer/2022022006/5ac46ca07f8b9a220b8cd175/html5/thumbnails/34.jpg)
MARC
Monday 2 December 13
![Page 35: Catmandu - SWIBswib.org/swib13/slides/jahn_swib13_128.pdf · What is it? • a Perl library • a command line tool • to import, transform and export (library) data • in a pragmatic](https://reader031.vdocument.in/reader031/viewer/2022022006/5ac46ca07f8b9a220b8cd175/html5/thumbnails/35.jpg)
Data
Monday 2 December 13
![Page 36: Catmandu - SWIBswib.org/swib13/slides/jahn_swib13_128.pdf · What is it? • a Perl library • a command line tool • to import, transform and export (library) data • in a pragmatic](https://reader031.vdocument.in/reader031/viewer/2022022006/5ac46ca07f8b9a220b8cd175/html5/thumbnails/36.jpg)
Data
Monday 2 December 13
![Page 37: Catmandu - SWIBswib.org/swib13/slides/jahn_swib13_128.pdf · What is it? • a Perl library • a command line tool • to import, transform and export (library) data • in a pragmatic](https://reader031.vdocument.in/reader031/viewer/2022022006/5ac46ca07f8b9a220b8cd175/html5/thumbnails/37.jpg)
Syntax
Monday 2 December 13
![Page 38: Catmandu - SWIBswib.org/swib13/slides/jahn_swib13_128.pdf · What is it? • a Perl library • a command line tool • to import, transform and export (library) data • in a pragmatic](https://reader031.vdocument.in/reader031/viewer/2022022006/5ac46ca07f8b9a220b8cd175/html5/thumbnails/38.jpg)
Syntax
title: War and peace
Monday 2 December 13
![Page 39: Catmandu - SWIBswib.org/swib13/slides/jahn_swib13_128.pdf · What is it? • a Perl library • a command line tool • to import, transform and export (library) data • in a pragmatic](https://reader031.vdocument.in/reader031/viewer/2022022006/5ac46ca07f8b9a220b8cd175/html5/thumbnails/39.jpg)
Syntax
title: War and peace
year: 1952
Monday 2 December 13
![Page 40: Catmandu - SWIBswib.org/swib13/slides/jahn_swib13_128.pdf · What is it? • a Perl library • a command line tool • to import, transform and export (library) data • in a pragmatic](https://reader031.vdocument.in/reader031/viewer/2022022006/5ac46ca07f8b9a220b8cd175/html5/thumbnails/40.jpg)
Syntax
title: War and peace
year: 1952
author: first: Lev Nikolaevič last: Tolstoj
Monday 2 December 13
![Page 41: Catmandu - SWIBswib.org/swib13/slides/jahn_swib13_128.pdf · What is it? • a Perl library • a command line tool • to import, transform and export (library) data • in a pragmatic](https://reader031.vdocument.in/reader031/viewer/2022022006/5ac46ca07f8b9a220b8cd175/html5/thumbnails/41.jpg)
Task* Use the RUG01 collection. Find the MARC fields for:
* title* language* subject* isbn* issn* extent (number of pages)* issued (the year of publication)* publication type* authors* publisher
* Write down any operations that are need to get an exact answer.
* Hint: http://www.loc.gov/marc/bibliographic/
Monday 2 December 13
![Page 42: Catmandu - SWIBswib.org/swib13/slides/jahn_swib13_128.pdf · What is it? • a Perl library • a command line tool • to import, transform and export (library) data • in a pragmatic](https://reader031.vdocument.in/reader031/viewer/2022022006/5ac46ca07f8b9a220b8cd175/html5/thumbnails/42.jpg)
Task
* Write a Catmandu Fix to extract all the fields from the example RUG01 records
Monday 2 December 13
![Page 43: Catmandu - SWIBswib.org/swib13/slides/jahn_swib13_128.pdf · What is it? • a Perl library • a command line tool • to import, transform and export (library) data • in a pragmatic](https://reader031.vdocument.in/reader031/viewer/2022022006/5ac46ca07f8b9a220b8cd175/html5/thumbnails/43.jpg)
Linked Data
Monday 2 December 13
![Page 44: Catmandu - SWIBswib.org/swib13/slides/jahn_swib13_128.pdf · What is it? • a Perl library • a command line tool • to import, transform and export (library) data • in a pragmatic](https://reader031.vdocument.in/reader031/viewer/2022022006/5ac46ca07f8b9a220b8cd175/html5/thumbnails/44.jpg)
Monday 2 December 13
![Page 45: Catmandu - SWIBswib.org/swib13/slides/jahn_swib13_128.pdf · What is it? • a Perl library • a command line tool • to import, transform and export (library) data • in a pragmatic](https://reader031.vdocument.in/reader031/viewer/2022022006/5ac46ca07f8b9a220b8cd175/html5/thumbnails/45.jpg)
http://hochstenbach.wordpress.com
“Daily doodles, sketches and cartoons” http://liesbethdestercke.tumblr.com/
Monday 2 December 13
![Page 46: Catmandu - SWIBswib.org/swib13/slides/jahn_swib13_128.pdf · What is it? • a Perl library • a command line tool • to import, transform and export (library) data • in a pragmatic](https://reader031.vdocument.in/reader031/viewer/2022022006/5ac46ca07f8b9a220b8cd175/html5/thumbnails/46.jpg)
http://hochstenbach.wordpress.com
“Daily doodles, sketches and cartoons” http://liesbethdestercke.tumblr.com/
about
title likes
Monday 2 December 13
![Page 47: Catmandu - SWIBswib.org/swib13/slides/jahn_swib13_128.pdf · What is it? • a Perl library • a command line tool • to import, transform and export (library) data • in a pragmatic](https://reader031.vdocument.in/reader031/viewer/2022022006/5ac46ca07f8b9a220b8cd175/html5/thumbnails/47.jpg)
nd cartoons” http://liesbethdestercke.tumblr.com/
likes
“Liesbeth De Stercke”
Monday 2 December 13
![Page 48: Catmandu - SWIBswib.org/swib13/slides/jahn_swib13_128.pdf · What is it? • a Perl library • a command line tool • to import, transform and export (library) data • in a pragmatic](https://reader031.vdocument.in/reader031/viewer/2022022006/5ac46ca07f8b9a220b8cd175/html5/thumbnails/48.jpg)
nd cartoons” http://liesbethdestercke.tumblr.com/
likes
“Liesbeth De Stercke”
about
title likes
Monday 2 December 13
![Page 49: Catmandu - SWIBswib.org/swib13/slides/jahn_swib13_128.pdf · What is it? • a Perl library • a command line tool • to import, transform and export (library) data • in a pragmatic](https://reader031.vdocument.in/reader031/viewer/2022022006/5ac46ca07f8b9a220b8cd175/html5/thumbnails/49.jpg)
...add image of that bubble network here...
Monday 2 December 13
![Page 50: Catmandu - SWIBswib.org/swib13/slides/jahn_swib13_128.pdf · What is it? • a Perl library • a command line tool • to import, transform and export (library) data • in a pragmatic](https://reader031.vdocument.in/reader031/viewer/2022022006/5ac46ca07f8b9a220b8cd175/html5/thumbnails/50.jpg)
RDF
Monday 2 December 13
![Page 51: Catmandu - SWIBswib.org/swib13/slides/jahn_swib13_128.pdf · What is it? • a Perl library • a command line tool • to import, transform and export (library) data • in a pragmatic](https://reader031.vdocument.in/reader031/viewer/2022022006/5ac46ca07f8b9a220b8cd175/html5/thumbnails/51.jpg)
TripleTriple
http://hochstenbach.wordpress.com http://purl.org/dc/elements/1.1/creator “Patrick Hochstenbach”
subject predicate object
Monday 2 December 13
![Page 52: Catmandu - SWIBswib.org/swib13/slides/jahn_swib13_128.pdf · What is it? • a Perl library • a command line tool • to import, transform and export (library) data • in a pragmatic](https://reader031.vdocument.in/reader031/viewer/2022022006/5ac46ca07f8b9a220b8cd175/html5/thumbnails/52.jpg)
http://hochstenbach.wordpress.com http://purl.org/dc/elements/1.1/creator “Patrick Hochstenbach”
subject predicate object
http://hochstenbach.wordpress.com http://purl.org/dc/elements/1.1/title “Daily doodles, sketches and cartoons”
Triple
http://liesbethdestercke.tumblr.com/ http://purl.org/dc/elements/1.1/creator “Liesbeth De Stercke”
http://liesbethdestercke.tumblr.com/ http://purl.org/dc/elements/1.1/title “Liesbeth De Stercke”
Monday 2 December 13
![Page 53: Catmandu - SWIBswib.org/swib13/slides/jahn_swib13_128.pdf · What is it? • a Perl library • a command line tool • to import, transform and export (library) data • in a pragmatic](https://reader031.vdocument.in/reader031/viewer/2022022006/5ac46ca07f8b9a220b8cd175/html5/thumbnails/53.jpg)
Vocabulary
Author
Creator
Main Entry - Personal Name
100-$$a
Monday 2 December 13
![Page 54: Catmandu - SWIBswib.org/swib13/slides/jahn_swib13_128.pdf · What is it? • a Perl library • a command line tool • to import, transform and export (library) data • in a pragmatic](https://reader031.vdocument.in/reader031/viewer/2022022006/5ac46ca07f8b9a220b8cd175/html5/thumbnails/54.jpg)
Vocabulary
Author
Creator
Main Entry - Personal Name
100-$$a
http://purl.org/dc/elements/1.1/
http://patrick.com/patricks/vocabulary http://www.loc.gov/marc/bibliographic/
http://wwww.iso.org/ISO-2709:2008
Monday 2 December 13
![Page 55: Catmandu - SWIBswib.org/swib13/slides/jahn_swib13_128.pdf · What is it? • a Perl library • a command line tool • to import, transform and export (library) data • in a pragmatic](https://reader031.vdocument.in/reader031/viewer/2022022006/5ac46ca07f8b9a220b8cd175/html5/thumbnails/55.jpg)
Task
* Write down the personal information about yourself from YAML into atabular form subject,predicate, object.
* Write all the subjects and predicates in the form of a URL.
* Create linked data pointing to the personal information of others.
Monday 2 December 13
![Page 56: Catmandu - SWIBswib.org/swib13/slides/jahn_swib13_128.pdf · What is it? • a Perl library • a command line tool • to import, transform and export (library) data • in a pragmatic](https://reader031.vdocument.in/reader031/viewer/2022022006/5ac46ca07f8b9a220b8cd175/html5/thumbnails/56.jpg)
Serialization
Monday 2 December 13
![Page 57: Catmandu - SWIBswib.org/swib13/slides/jahn_swib13_128.pdf · What is it? • a Perl library • a command line tool • to import, transform and export (library) data • in a pragmatic](https://reader031.vdocument.in/reader031/viewer/2022022006/5ac46ca07f8b9a220b8cd175/html5/thumbnails/57.jpg)
RDF/XML
<?xml version="1.0"?><rdf:RDF xmlns:rdf="http://www.w3.org/1999/02/22-rdf-syntax-ns#" xmlns:wgspos="http://www.w3.org/2003/01/geo/wgs84_pos#" xmlns:ns="http://purl.org/dc/elements/1.1/" xmlns:ns1="http://xmlns.com/foaf/0.1/"> <rdf:Description rdf:about="htpp://hochstenbach.wordpress.com"> <ns:title xml:lang="en">Doodles</ns:title> <wgspos:location wgspos:lat="9.93492" wgspos:long="51.539371" /> <ns1:age rdf:datatype="http://www.w3.org/2001/XMLSchema#integer">42</ns1:age> <ns1:workplaceHomepage rdf:resource="http://lib.ugent.be/" /> </rdf:Description></rdf:RDF>
Monday 2 December 13
![Page 58: Catmandu - SWIBswib.org/swib13/slides/jahn_swib13_128.pdf · What is it? • a Perl library • a command line tool • to import, transform and export (library) data • in a pragmatic](https://reader031.vdocument.in/reader031/viewer/2022022006/5ac46ca07f8b9a220b8cd175/html5/thumbnails/58.jpg)
RDF/Turtle
@prefix dc: <http://purl.org/dc/elements/1.1/> .@prefix foaf: <hrrp://xmlns.com/foaf/0.1/>.
<htpp://hochstenbach.wordpress.com> dc:title "Doodles"@en ; geo:location [ geo:lat “"9.93492" ; geo:long “51.539371"
] ; foaf:age 42 ; foaf:workplaceHomepage <http://lib.ugent.be/> .
Monday 2 December 13
![Page 59: Catmandu - SWIBswib.org/swib13/slides/jahn_swib13_128.pdf · What is it? • a Perl library • a command line tool • to import, transform and export (library) data • in a pragmatic](https://reader031.vdocument.in/reader031/viewer/2022022006/5ac46ca07f8b9a220b8cd175/html5/thumbnails/59.jpg)
aRDF
---'_id': htpp://hochstenbach.wordpress.comdc:title: Doodles@enfoaf:age: 42^^xsd:integerfoaf:workplaceHomepage: '@id': http://lib.ugent.begeo:location: geo:lat: 9.93492 geo:long: 51.539371
Monday 2 December 13
![Page 60: Catmandu - SWIBswib.org/swib13/slides/jahn_swib13_128.pdf · What is it? • a Perl library • a command line tool • to import, transform and export (library) data • in a pragmatic](https://reader031.vdocument.in/reader031/viewer/2022022006/5ac46ca07f8b9a220b8cd175/html5/thumbnails/60.jpg)
Turtle
Monday 2 December 13
![Page 61: Catmandu - SWIBswib.org/swib13/slides/jahn_swib13_128.pdf · What is it? • a Perl library • a command line tool • to import, transform and export (library) data • in a pragmatic](https://reader031.vdocument.in/reader031/viewer/2022022006/5ac46ca07f8b9a220b8cd175/html5/thumbnails/61.jpg)
Triple
http://hochstenbach.wordpress.com http://purl.org/dc/elements/1.1/creator “Patrick Hochstenbach”
<http://hochstenbach.wordpress.com>
subject predicate object
<http://purl.org/dc/elements/1.1/creator> “Patrick Hochstenbach” .
<http://hochstenbach.wordpress.com>
<http://purl.org/dc/elements/1.1/creator> “Patrick Hochstenbach” .
Monday 2 December 13
![Page 62: Catmandu - SWIBswib.org/swib13/slides/jahn_swib13_128.pdf · What is it? • a Perl library • a command line tool • to import, transform and export (library) data • in a pragmatic](https://reader031.vdocument.in/reader031/viewer/2022022006/5ac46ca07f8b9a220b8cd175/html5/thumbnails/62.jpg)
Prefix
http://hochstenbach.wordpress.com http://purl.org/dc/elements/1.1/creator “Patrick Hochstenbach”
<http://hochstenbach.wordpress.com>
subject predicate object
dc:creator “Patrick Hochstenbach” .
@prefix dc: <http://purl.org/dc/elements/1.1> .
Monday 2 December 13
![Page 63: Catmandu - SWIBswib.org/swib13/slides/jahn_swib13_128.pdf · What is it? • a Perl library • a command line tool • to import, transform and export (library) data • in a pragmatic](https://reader031.vdocument.in/reader031/viewer/2022022006/5ac46ca07f8b9a220b8cd175/html5/thumbnails/63.jpg)
Subjects “;”
http://hochstenbach.wordpress.com http://purl.org/dc/elements/1.1/creator “Patrick Hochstenbach”
<http://hochstenbach.wordpress.com>
subject predicate object
dc:creator “Patrick Hochstenbach” .
@prefix dc: <http://purl.org/dc/elements/1.1> .
http://hochstenbach.wordpress.com http://purl.org/dc/elements/1.1/title “Daily doodles, sketches and cartoons”
<http://hochstenbach.wordpress.com> dc:title “Daily doodles, sketches and cartoons” .
Monday 2 December 13
![Page 64: Catmandu - SWIBswib.org/swib13/slides/jahn_swib13_128.pdf · What is it? • a Perl library • a command line tool • to import, transform and export (library) data • in a pragmatic](https://reader031.vdocument.in/reader031/viewer/2022022006/5ac46ca07f8b9a220b8cd175/html5/thumbnails/64.jpg)
Subjects “;”
http://hochstenbach.wordpress.com http://purl.org/dc/elements/1.1/creator “Patrick Hochstenbach”
<http://hochstenbach.wordpress.com>
subject predicate object
dc:creator “Patrick Hochstenbach” ;
@prefix dc: <http://purl.org/dc/elements/1.1> .
http://hochstenbach.wordpress.com http://purl.org/dc/elements/1.1/title “Daily doodles, sketches and cartoons”
dc:title “Daily doodles, sketches and cartoons” .
Monday 2 December 13
![Page 65: Catmandu - SWIBswib.org/swib13/slides/jahn_swib13_128.pdf · What is it? • a Perl library • a command line tool • to import, transform and export (library) data • in a pragmatic](https://reader031.vdocument.in/reader031/viewer/2022022006/5ac46ca07f8b9a220b8cd175/html5/thumbnails/65.jpg)
Objects “,”
http://hochstenbach.wordpress.com http://purl.org/dc/elements/1.1/creator “Patrick Hochstenbach”
<http://hochstenbach.wordpress.com>
subject predicate object
dc:creator “Patrick Hochstenbach” ;
@prefix dc: <http://purl.org/dc/elements/1.1> .
http://hochstenbach.wordpress.com http://purl.org/dc/elements/1.1/title “Daily doodles, sketches and cartoons”
dc:title “Daily doodles, sketches and cartoons” ,
http://hochstenbach.wordpress.com http://purl.org/dc/elements/1.1/title “Hochstenbach”
“Hochstenbach” .
Monday 2 December 13
![Page 66: Catmandu - SWIBswib.org/swib13/slides/jahn_swib13_128.pdf · What is it? • a Perl library • a command line tool • to import, transform and export (library) data • in a pragmatic](https://reader031.vdocument.in/reader031/viewer/2022022006/5ac46ca07f8b9a220b8cd175/html5/thumbnails/66.jpg)
Task
* Write your personal information from the tabular format into the Turtle language.
* Validate your Turtle at http://www.rdfabout.com/demo/validator/
Monday 2 December 13
![Page 67: Catmandu - SWIBswib.org/swib13/slides/jahn_swib13_128.pdf · What is it? • a Perl library • a command line tool • to import, transform and export (library) data • in a pragmatic](https://reader031.vdocument.in/reader031/viewer/2022022006/5ac46ca07f8b9a220b8cd175/html5/thumbnails/67.jpg)
aRDF
Monday 2 December 13
![Page 68: Catmandu - SWIBswib.org/swib13/slides/jahn_swib13_128.pdf · What is it? • a Perl library • a command line tool • to import, transform and export (library) data • in a pragmatic](https://reader031.vdocument.in/reader031/viewer/2022022006/5ac46ca07f8b9a220b8cd175/html5/thumbnails/68.jpg)
Literals
<http://hochstenbach.wordpress.com>
dc:title “Daily doodles, sketches and cartoons” .
@prefix dc: <http://purl.org/dc/elements/1.1/> .
_id: http://hochstenbach.wordpress.comdc:title: “Daily doodles, sketches and cartoons”
add_field(‘_id’,’htpp://hochstenbach.wordpress.com’);add_field(‘dc:title’,’Daily doodles, sketches and cartoons’);
http://dublincore.org/documents/dcmi-terms/
Monday 2 December 13
![Page 69: Catmandu - SWIBswib.org/swib13/slides/jahn_swib13_128.pdf · What is it? • a Perl library • a command line tool • to import, transform and export (library) data • in a pragmatic](https://reader031.vdocument.in/reader031/viewer/2022022006/5ac46ca07f8b9a220b8cd175/html5/thumbnails/69.jpg)
<http://hochstenbach.wordpress.com>
dc:title “Daily doodles, sketches and cartoons”@en.
@prefix dc: <http://purl.org/dc/elements/1.1/> .
_id: http://hochstenbach.wordpress.comdc:title: “Daily doodles, sketches and cartoons@en”
add_field(‘_id’,‘http://hochstenbach.wordpress.com’);add_field(‘dc:title’,’Daily doodles, sketches and cartoons@en’);
Language
Monday 2 December 13
![Page 70: Catmandu - SWIBswib.org/swib13/slides/jahn_swib13_128.pdf · What is it? • a Perl library • a command line tool • to import, transform and export (library) data • in a pragmatic](https://reader031.vdocument.in/reader031/viewer/2022022006/5ac46ca07f8b9a220b8cd175/html5/thumbnails/70.jpg)
Numbers
<http://hochstenbach.wordpress.com>
foaf:age “42”^^xsd:integer .
@prefix foaf: <http://xmlns.com/foaf/0.1/> .
_id: http://hochstenbach.wordpress.comfoaf:age: 42^^xsd:integer
add_field(‘_id’,’htpp://hochstenbach.wordpress.com’);add_field(‘foaf:age’,’42^^xsd:integer’);
http://xmlns.com/foaf/spec/
Monday 2 December 13
![Page 71: Catmandu - SWIBswib.org/swib13/slides/jahn_swib13_128.pdf · What is it? • a Perl library • a command line tool • to import, transform and export (library) data • in a pragmatic](https://reader031.vdocument.in/reader031/viewer/2022022006/5ac46ca07f8b9a220b8cd175/html5/thumbnails/71.jpg)
XSD Data Types
• xsd:string , xsd:language
• xsd:date , xsd:time , xsd:dateTime , xsd:duration
• xsd:integer , xsd:float
http://www.w3schools.com/schema/schema_dtypes_date.asp
Monday 2 December 13
![Page 72: Catmandu - SWIBswib.org/swib13/slides/jahn_swib13_128.pdf · What is it? • a Perl library • a command line tool • to import, transform and export (library) data • in a pragmatic](https://reader031.vdocument.in/reader031/viewer/2022022006/5ac46ca07f8b9a220b8cd175/html5/thumbnails/72.jpg)
URI Reference
<http://hochstenbach.wordpress.com>
foaf:workplaceHomepage <http://lib.ugent.be>.
@prefix foaf: <http://xmlns.com/foaf/0.1/> .
_id: http://hochstenbach.wordpress.comfoaf:workplaceHomepage: http://lib.ugent.be
add_field(‘_id’,’htpp://hochstenbach.wordpress.com’);add_field(‘foaf:workplaceHomepage’,’http://lib.ugent.be’);
http://xmlns.com/foaf/spec/
Monday 2 December 13
![Page 73: Catmandu - SWIBswib.org/swib13/slides/jahn_swib13_128.pdf · What is it? • a Perl library • a command line tool • to import, transform and export (library) data • in a pragmatic](https://reader031.vdocument.in/reader031/viewer/2022022006/5ac46ca07f8b9a220b8cd175/html5/thumbnails/73.jpg)
Blank Node
<http://hochstenbach.wordpress.com>
geo:location _:blabla.
@prefix geo: <http://www.w3.org/2003/01/geo/wgs84_pos#> .
_:blablageo:lat “51.0500” ;
geo:long “3.7167” .
_id: http://hochstenbach.wordpress.comgeo:location.geo:lat: 51.0500geo:location.geo:long: 3.7167
add_field(‘_id’,’htpp://hochstenbach.wordpress.com’);add_field(‘geo:location.geo:lat’,’51.0500’);add_field(‘geo:location.geo:long’,’3.7167’);
Monday 2 December 13
![Page 74: Catmandu - SWIBswib.org/swib13/slides/jahn_swib13_128.pdf · What is it? • a Perl library • a command line tool • to import, transform and export (library) data • in a pragmatic](https://reader031.vdocument.in/reader031/viewer/2022022006/5ac46ca07f8b9a220b8cd175/html5/thumbnails/74.jpg)
Class
<http://hochstenbach.wordpress.com> a foaf:Person .
@prefix foaf: <http://xmlns.com/foaf/0.1/> .
_id: http://hochstenbach.wordpress.coma: foaf:Person
add_field(‘_id’,’htpp://hochstenbach.wordpress.com’);add_field(‘a’,’foaf:Person’);
http://code.google.com/p/bibotools/source/browse/bibo-ontology/tags/1.0/bibo.n3
Monday 2 December 13
![Page 75: Catmandu - SWIBswib.org/swib13/slides/jahn_swib13_128.pdf · What is it? • a Perl library • a command line tool • to import, transform and export (library) data • in a pragmatic](https://reader031.vdocument.in/reader031/viewer/2022022006/5ac46ca07f8b9a220b8cd175/html5/thumbnails/75.jpg)
Task
@prefix dc: <http://purl.org/dc/elements/1.1/> . <http://swib.org> dc:title “Semantic Web in Libraries” .
* Translate the Turtle below in aRDF
Monday 2 December 13
![Page 76: Catmandu - SWIBswib.org/swib13/slides/jahn_swib13_128.pdf · What is it? • a Perl library • a command line tool • to import, transform and export (library) data • in a pragmatic](https://reader031.vdocument.in/reader031/viewer/2022022006/5ac46ca07f8b9a220b8cd175/html5/thumbnails/76.jpg)
Task* Use Mongo Admin Test to create the following Turtle expression:
@prefix dc: <http://purl.org/dc/elements/1.1/> . <http://swib.org> dc:title “Semantic Web in Libraries” .
* Add code to specify this is an English title
* Add a title in another language
* Add the number of times you attended SWIB in dc:extent
* Create an integer value out of dc:extent
* Classify swib.org as a FOAF ‘Organization’
* Express that SWIB is a member of the HBZ http://www.hbz-nrw.de/
Monday 2 December 13
![Page 77: Catmandu - SWIBswib.org/swib13/slides/jahn_swib13_128.pdf · What is it? • a Perl library • a command line tool • to import, transform and export (library) data • in a pragmatic](https://reader031.vdocument.in/reader031/viewer/2022022006/5ac46ca07f8b9a220b8cd175/html5/thumbnails/77.jpg)
Task
https://wiki1.hbz-nrw.de/display/SEM/Converting+the+Open+Data+from+the+hbz+to+BIBO
Convert the rug01 MARC records to RDF using
as example
http://www.loc.gov/marc/bibliographic/
Hint: translate the mapping to MARC
Monday 2 December 13
![Page 78: Catmandu - SWIBswib.org/swib13/slides/jahn_swib13_128.pdf · What is it? • a Perl library • a command line tool • to import, transform and export (library) data • in a pragmatic](https://reader031.vdocument.in/reader031/viewer/2022022006/5ac46ca07f8b9a220b8cd175/html5/thumbnails/78.jpg)
Linked Data
Monday 2 December 13
![Page 79: Catmandu - SWIBswib.org/swib13/slides/jahn_swib13_128.pdf · What is it? • a Perl library • a command line tool • to import, transform and export (library) data • in a pragmatic](https://reader031.vdocument.in/reader031/viewer/2022022006/5ac46ca07f8b9a220b8cd175/html5/thumbnails/79.jpg)
cmp_field
marc_map(‘008/7-10’,‘year’);cmp_field('year', '1990');
• year == 1 if year > 1900
• year == 0 if year == 1900
• year == -1 if year < 1900
Monday 2 December 13
![Page 80: Catmandu - SWIBswib.org/swib13/slides/jahn_swib13_128.pdf · What is it? • a Perl library • a command line tool • to import, transform and export (library) data • in a pragmatic](https://reader031.vdocument.in/reader031/viewer/2022022006/5ac46ca07f8b9a220b8cd175/html5/thumbnails/80.jpg)
count
add_field(‘author.$append’,‘James’);add_field(‘author.$append’,‘Jones’);count('author');
author == 2
Monday 2 December 13
![Page 81: Catmandu - SWIBswib.org/swib13/slides/jahn_swib13_128.pdf · What is it? • a Perl library • a command line tool • to import, transform and export (library) data • in a pragmatic](https://reader031.vdocument.in/reader031/viewer/2022022006/5ac46ca07f8b9a220b8cd175/html5/thumbnails/81.jpg)
weave_by_id
weave_by_id(‘cover’);
lookup contains the complete record from the store ‘covers’ where ‘_id’ is the
current record id
Monday 2 December 13
![Page 82: Catmandu - SWIBswib.org/swib13/slides/jahn_swib13_128.pdf · What is it? • a Perl library • a command line tool • to import, transform and export (library) data • in a pragmatic](https://reader031.vdocument.in/reader031/viewer/2022022006/5ac46ca07f8b9a220b8cd175/html5/thumbnails/82.jpg)
weave_by_query
add_field('lookup.name','Jerrold Katz');weave_by_query('lookup', -store=>'author');
lookup contains the complete record from the store ‘author’ where ‘name’ is
‘Jerrold Katz’
Monday 2 December 13
![Page 83: Catmandu - SWIBswib.org/swib13/slides/jahn_swib13_128.pdf · What is it? • a Perl library • a command line tool • to import, transform and export (library) data • in a pragmatic](https://reader031.vdocument.in/reader031/viewer/2022022006/5ac46ca07f8b9a220b8cd175/html5/thumbnails/83.jpg)
Task
* Find for some RUG01 records the URL to a cover image
* Create a YAML file in Notepad containing the ‘_id’ of the RUG01 record and the ‘cover_remote’ URL to the image
* Upload the YAML file into the cover database
* Use weave_by_id to test insert the image into the record
* Find an appropriate RDF expression for this URL
Monday 2 December 13
![Page 84: Catmandu - SWIBswib.org/swib13/slides/jahn_swib13_128.pdf · What is it? • a Perl library • a command line tool • to import, transform and export (library) data • in a pragmatic](https://reader031.vdocument.in/reader031/viewer/2022022006/5ac46ca07f8b9a220b8cd175/html5/thumbnails/84.jpg)
Task
* Find for some RUG01 record the author name in Wikipedia (or any other authoritative page)
* Create a YAML file in Notepad containing the author ‘name’ and ‘url’ the his website
* Upload the YAML file in the author database
* Use weave_by_query to lookup the author name for the record
* Find an appropriate RDF expression for this URL
Monday 2 December 13