![Page 1: Presentation & discussion · Presentation & discussion EURISCO training workshop, 12th to 14th October 2016, Angers, France . EURISCO intranet I • Development of new import component](https://reader034.vdocument.in/reader034/viewer/2022052004/6017841c4c5af4321c0cfdaa/html5/thumbnails/1.jpg)
Stephan Weise 13 October 2016
Passport data updates
Presentation & discussion
EURISCO training workshop, 12th to 14th October 2016, Angers, France
![Page 2: Presentation & discussion · Presentation & discussion EURISCO training workshop, 12th to 14th October 2016, Angers, France . EURISCO intranet I • Development of new import component](https://reader034.vdocument.in/reader034/viewer/2022052004/6017841c4c5af4321c0cfdaa/html5/thumbnails/2.jpg)
EURISCO intranet I
• Development of new import component for NIs
– Web interface with Oracle APEX
– PL/SQL packages for uploading, checking and updating data
– Implementation of incremental updates
![Page 3: Presentation & discussion · Presentation & discussion EURISCO training workshop, 12th to 14th October 2016, Angers, France . EURISCO intranet I • Development of new import component](https://reader034.vdocument.in/reader034/viewer/2022052004/6017841c4c5af4321c0cfdaa/html5/thumbnails/3.jpg)
EURISCO intranet II
New Java-
based
importer
![Page 4: Presentation & discussion · Presentation & discussion EURISCO training workshop, 12th to 14th October 2016, Angers, France . EURISCO intranet I • Development of new import component](https://reader034.vdocument.in/reader034/viewer/2022052004/6017841c4c5af4321c0cfdaa/html5/thumbnails/4.jpg)
Java-based import I
• So far: Web-based upload – Upload via tab-separated text file, UTF-8 encoded
– Often problems with columns separators + character encoding
• Now: Java-based Excel import – Summarises file upload + data import
– User gets informed when the integrity checks are finished
![Page 5: Presentation & discussion · Presentation & discussion EURISCO training workshop, 12th to 14th October 2016, Angers, France . EURISCO intranet I • Development of new import component](https://reader034.vdocument.in/reader034/viewer/2022052004/6017841c4c5af4321c0cfdaa/html5/thumbnails/5.jpg)
Java-based import II
• JRE 1.8
• Java WS
• Oracle standard
port 1521 enabled
![Page 6: Presentation & discussion · Presentation & discussion EURISCO training workshop, 12th to 14th October 2016, Angers, France . EURISCO intranet I • Development of new import component](https://reader034.vdocument.in/reader034/viewer/2022052004/6017841c4c5af4321c0cfdaa/html5/thumbnails/6.jpg)
Integrity checks I
![Page 7: Presentation & discussion · Presentation & discussion EURISCO training workshop, 12th to 14th October 2016, Angers, France . EURISCO intranet I • Development of new import component](https://reader034.vdocument.in/reader034/viewer/2022052004/6017841c4c5af4321c0cfdaa/html5/thumbnails/7.jpg)
Integrity checks II – error report
![Page 8: Presentation & discussion · Presentation & discussion EURISCO training workshop, 12th to 14th October 2016, Angers, France . EURISCO intranet I • Development of new import component](https://reader034.vdocument.in/reader034/viewer/2022052004/6017841c4c5af4321c0cfdaa/html5/thumbnails/8.jpg)
Integrity checks III - example
• Wrong
date
format
![Page 9: Presentation & discussion · Presentation & discussion EURISCO training workshop, 12th to 14th October 2016, Angers, France . EURISCO intranet I • Development of new import component](https://reader034.vdocument.in/reader034/viewer/2022052004/6017841c4c5af4321c0cfdaa/html5/thumbnails/9.jpg)
Integrity checks IV - example
• Invalid or
multiple
donor
codes
![Page 10: Presentation & discussion · Presentation & discussion EURISCO training workshop, 12th to 14th October 2016, Angers, France . EURISCO intranet I • Development of new import component](https://reader034.vdocument.in/reader034/viewer/2022052004/6017841c4c5af4321c0cfdaa/html5/thumbnails/10.jpg)
Incremental updates I
• Why incremental updates?
– So far: Full replacement
• Delete whole dataset + reimport data afterwards
• Even if only a couple of rows have been modified
• Not possible to update parts of data (e.g. single genebank collection)
– That’s why: From full replacement to real update
• Only incremental data needs to be updated
• Necessary: Unique identifiers
• Currently: Combination of NICODE, INSTCODE, ACCENUMB and GENUS
– Important for managing C&E data
• Cannot exist without passport data
![Page 11: Presentation & discussion · Presentation & discussion EURISCO training workshop, 12th to 14th October 2016, Angers, France . EURISCO intranet I • Development of new import component](https://reader034.vdocument.in/reader034/viewer/2022052004/6017841c4c5af4321c0cfdaa/html5/thumbnails/11.jpg)
Incremental updates II
• Deletion candidates
– Check of new
dataset against
existing data
– List of accessions
not contained in the
new dataset
– Not deleted
automatically
– False positive hits in
case of partial
update!!!
![Page 12: Presentation & discussion · Presentation & discussion EURISCO training workshop, 12th to 14th October 2016, Angers, France . EURISCO intranet I • Development of new import component](https://reader034.vdocument.in/reader034/viewer/2022052004/6017841c4c5af4321c0cfdaa/html5/thumbnails/12.jpg)
Final decision
![Page 13: Presentation & discussion · Presentation & discussion EURISCO training workshop, 12th to 14th October 2016, Angers, France . EURISCO intranet I • Development of new import component](https://reader034.vdocument.in/reader034/viewer/2022052004/6017841c4c5af4321c0cfdaa/html5/thumbnails/13.jpg)
Next steps (in background)
• Updated dataset will be applied to EURISCO stage
schema
• EURISCO stage will be synchronised to the EURISCO
web schema (Time lag!)
– Not in main business hours
– Rebuild of materialised views
– Creation of new full dump (MS Access)
– News message on EURISCO webpage
Page 13