blending and deblending data in the daily routine of a university...

ELAGConference2018|WolfgangStille |UniversityandStateLibrary@TUDarmstadt|1

Blendinganddeblendingdatainthedailyroutineofauniversitylibrary


BlendingandDeblending

1)Named Entity

4)Ingest to SOLRindex

SELECT ?s WHERE {{?s rdfs:label “Olea Europaea"@en;a owl:Thing . } UNION { ?altName rdfs:label "Olea Europaea"@en;dbo:wikiPageRedirects ?s .}

[…]}

SPARQLEndpoint2)

rdfs:label: Olivedbo:family: Oleaceaedbo:genus: Oleadbo:order: Lamialesdbo:abstract: The olive /ˈɒlɪv/ or /ˈɑːləv/, known by the botanical name Olea europaea, meaning "european olive", (syn. Olea sylvestris) is a species of small tree in the family Oleaceae, found in much ofAfrica, the Mediterranean Basin fromPortugal to the Levant, [...]foaf:depiction:

3)Blendwith attributes fromLOD


PartI:StatusQuoofData(De-)Blending

ILS

IDM

Reporting

Digitization

Discovery

Biblio

OA

RIS

CBS/WorldCat

RDM

Presentation

RFIDInv

PM/Collaboration

ArchivingCMS

Enrichment /LOD

PICALBS3


Bymeansofwhatsoftware?

Commercial• PICALBSversion 3and 4• PICACBS• NovellNetIQ• ExLibris Rosetta• Atlassian Confluence /JIRA/Crowd• FirstSpirit CMS• VisualLibrary

OpenSource• Eprints• Dspace• OJS• vufind• mySQL,PostgreSQL,SOLR,Tomcat• Dwork• Catmandu• OpenRefine• Thousands of lines of Perl/PHP/Python/JAVA

code


HowintegratedareIntegratedLibrarySystems?

IDM

Reporting

Digitization

Discovery

Biblio

OA

RIS

CBS/WorldCat

RDM

Collaboration

RFIDInv

PM

ArchivingCMS

Enrichment /LOD

ILS

PICALBS3


Wheretogo?

• RequirementstoILShavechangedsignificantlyovertheyears:— Librariesaresupposedtoofferalotmoredigitalservicesthaninthepast

— Serviceshavetointerconnecttootherservicesoncampus,e.g.IDM,CampusManagement,ResearchInformationSystem,ResearchDataRepository,…

— Servicesshouldbereusableinothercontextsandinteroperable

• ThisisonlypoorlyaddressedbycommercialILSproducts:

— Duringthelasttwoyears,wehadaworkinggrouprunninganassessmentoftwocommercialILSproducts.Theresultwas,thatnoneofthesystemsattheiractualstatesatisfiedour(quitebasic)requirements.


ConventionalMigration:ReplaceILS?

ILSAcquisition

LoanCataloguing

IDM

Reporting

Digitization

Discovery

Biblio

OA

RIS

CBS/WorldCat

RDM

Collaboration

RFIDInv

PM

ArchivingCMS

Enrichment /LOD


ConventionalMigration:Onlyfewmodulesavailable,resthastobeconnectedbyus…

ILSAcquisition

LoanCataloguing

IDM

Reporting

Digitization

Discovery

Biblio

OA

RIS

CBS/WorldCat

RDM

Collaboration

RFIDInv

PM

ArchivingCMS

Enrichment /LOD


AlternativeTopology:Decentralize,Modulize

IDM

Reporting

Digitization

Discovery

Biblio

OA RIS

Acquisition

RDM RFIDInvPM

Archiving

MessageBus/APILayer

Loan ERM

Enrichment /LOD

…

External DataSources


StepwiseMigrationofServices

IDMBiblio CBS/WorldCat

RDMPM

MessageBus/APILayer ILSAcquisition

LoanCataloguing

Digitization

Discovery

Presentation

PM/Collaboration

ArchivingCMS

Enrichment /LOD

Moveservices stepwise onto the APIlayer


PartII:CommercialSoftware

Benefitsofcommercial softwarefromamanagementperspective• Costsareeasytoestimate(aretheyreally?)

• Definedrangeoffunctionalityisdelivered(isit?)

• Notroublewithhumanresources,esp.permanentpositions(butalsonoinhouseknowhow)

• Professionalsupportfromthevendor(sometimes)

Difficultiesyoumayruninto• NDAs:youarenotallowedtotalkaboutdifficultieswithothercustomers

• Restrictedfunctionalityinsomeareas(à workaroundsnecessary)

• Yourrequirementsmaynotbefullyaddressedbythevendor(à workarounds)

• Vendorsupportcannot/doesn’thelpyou(à workarounds)

• Productportfoliochanges/productsaresoldordiscontinued/newproductsarenotdownwardcompatible/productcoststwicethepricenextyear


Examplesfromthelasttwoyearswithcommercialsoftwarevendors• ProjectA:LargeILSsoftwareevaluationprojectwascancelledafterseveralmonths,becausethesystemwasnotoperationaltorunalargelibrary

• ProjectB:Theadvertisedandofferedproductwasnotdevelopedatall,becausetherewasnomarketforit(afterfundsfromtheuniversityhavebeenacquired)

• ProjectC:Theprojectwasfinished1.5yearslaterthanplanned,becausetheprojectmanagerchanged3timesduringtheproject

• ProjectD:Theprojectisstillnotfinishedafter2yearsdelaybecauseoftechnicalissues

• ProjectE:Inhisrequirementsanalysis,thevendorstates50openquestionsandproblemsforourspecificlibrary,butnosolutions.Thishappenedinthepre-salesphase.

• tbc…


Motivation

https://www.flickr.com/photos/dff-jisc/3933622822


PossibleConsequences

Nowarrantythattheprojectwillendup

successfully.Ifnot,then:

§ Youmayloosetime

§ Youmayloosemoney

§ Youmaylooseboth

§ Youmayhaveapieceofsoftwarethatfeatures

onlyhalfoftherequirements,andyouhaveto

improvisetherestbyyourselfinanenvironment

youarenotfamiliarwith

§ Solutionofproblemsshiftsfromtheprojectlayer

tothemanagement layer.Theproblemcannot

besolveddirectly

§ Allofthisdoesn’tmakeyouhappy


WhataboutDIY?

https://xkcd.com/927/


DIYBestPractices

• Havingquestionsisnaturallyagoodthing

• Gettinginvolvedisalsoagoodthing

• Takingresponsibilitymightbeagoodthing(couldbealsoseenasadrawback)

Whatelse?

• Havealookatexistingsolutions/preventisolatedsolutions

• Talktopeoplewhouseand/ordevelopthatsoftwareinaproductivecontext

• Join/foundanopensourcesoftwareprojecttobenotalone

• Considerexistingstandards

• Acommunitynetworkisimportant(e.g.nationalconsortia)

Alsonowarrantythattheprojectwillendupsuccessfully,butyouareinchargeandyoucanact.


TheFutureof(De-)Blending?

• PlausibleArchitecture(API,microservices,…)

• Blendofcommunitydrivenprojectandcommercialplayerslookspromising

• Possibilitytogetinvolvedatdifferentlevelsandwithdifferentamountsofresources

• Manageablerisk


OurConsequence:CommitmentinFOLIO

• SinceMarch2018,ULBDarmstadtandanotherlibraryinHessespendaround1FTEinFOLIOdevelopment(ERMprojectdrivenbyGBV)

• AccessionoftheHessianLibraryFederationtotheOpenLibraryFoundationisdeterminedandwilltakeplacesoon

• AnotherFTEasprojectmanager/softwaredeveloperisactuallyadvertised

• AlluniversitylibrariesinHessemakecommitmentsinSpecialInterestGroups

• Commitmentisfixedforoneyear,afterthatevaluated,andhopefullycontinued…


Aspectstoconvincemanagement

• Youhavetododevelopmentanyways(eitherworkaroundsorpropersoftware)

• Competitionisgoodforbusiness:thereshouldbeatleastthreedifferentoptionstoavoidamonopoly

• WhataboutaTCOestimate?(noonedoes,atleast5-10yearsshouldbestandard)

• Attractionofexternalprojectfundingmaybepossible,atleasttemporarily

• Inadecentralizedarchitecture,therearemoreoptionstogetservicesdeveloped(OSS/commercialmodules,DIY,3rd partycommissionedwork,…)

• Humanresourcesdevelopmentcantakeplacefromthebeginningoftheproject(requirementsanalysis,implementation,testing,usage,…)

• ActiveDevelopmentofdigitalliteracyiscrucialfortoday’slibraries’staff

• Stayaslongaspossibleindependentfromvendorlock-insinyourdecisionmakingprocess


Wealreadydidthatinthepast!

ILSAcquisition

LoanCataloguing

IDM

Reporting

Digitization

Discovery

Biblio

OA

RIS

CBS/WorldCat

RDM

Collaboration

RFIDInv

PM

ArchivingCMS

Enrichment /LOD


Conclusion

• Thereisnoentirelyhappysolutionofftheshelfforfuturelibrarysystems

• Youhavetogetinvolvedanywaysregardlesswhichwayyouchoose

• Decentralizedarchitecturesaremorefuture-proofforscientificdigitallibrarybusinessthanmonolithicsystems

• Byparticipatingacommunityproject,youstrengthenthecommunityandyourdepartmentsexpertiseatthesametime

• Involvementcanbepositivelyusedtodevelophumanresourcestowardsmoredigitalliteracyaroundthelibrarystaff

Takingpartinthecookingprocessis

moreattractivethanjusteating

blending and deblending data in the daily routine of a university...

Documents