Supporting Cyberscholarship in American Social History: The DLF Aquifer Story Katherine KottWith content contributed by Susan Harum and Chick MarkleyCNI Task Force MeetingDecember 11, 2007

Focus on American Social History OnlineMellon funded project within DLF Aquifer initiativeOutcome to make digital collections in American culture and life easier for scholars to find and useAggregate content to enable multiple collections to be used as oneDeliver content through multiple channels

Whos whoAmerican Social History Online relies on Aquifer working groups, collaborative infrastructure in placeCore team focused on development and assessment, coordination with working groupsSusan Harum (UIUC) business analyst/assessment expertKat Hagedorn (Michigan) data analystChick Markley (Freewheeling contractor) systems architect/developerTom Habing (UIUC) developer

DLF Aquifer participant librariesBibliotheca AlexandrinaCalifornia Digital LibraryCornell UniversityEmory UniversityIndiana UniversityJohns Hopkins UniversityLibrary of CongressNew York UniversityStanford UniversityUniversity of Illinois, Urbana-ChampaignUniversity of MichiganUniversity of MinnesotaUniversity of Southern CaliforniaUniversity of TennesseeUniversity of VirginiaYale University

Overview of accomplishmentsArchitecturePortal with Zoteroinitial SEO analysisMODSGuidelinesLevels of adoptionMARC to MODS transformHarvesting workflowAgile development process

Website with Zotero in FirefoxRESERVED

How we got hereInitially unsure we would build a websiteSEO required for commercial search service scenarioZotero implementation requires websiteWebsite seemed like best way to present services (SRU) for Sakai and federated search scenariosWebsite could be used as cohesive way to present the projectIn June 2007, team set goal to have website ready for DLF Fall Forum in November 2007

Website developmentStarted with baseline functionality on Michigan MODS portalAdded and cross checked common business functions developed by SWG (integrated with DLF Services Framework)Architecture chosen to supportWebsite functionalitySakaiFederated search

TWG architectural framework

Architectural implementationToJSONjavaMODSmysqlJSONmysql to SOLRjavaRailsrubySOLR/LucenejavaHarvesterDLXSeXistjavato JSONjavabrowsersto eXistjavaservicesbots

A plug for Ruby on Rails from our systems architectIt's the frameworkDRY (don't repeat yourself)Tightly coupled testing harnessExuberant and dedicated community

Enter, Agile developmentSmall development team, distributed environmentShort timelineno time for traditional FRD processDesire within digital library community for lightweight process modelEmphasis on user driven developmentNot to be confused with final assessment activities for American Social History Online

ModelTimeframe: August 29 November 2, 2007 (8 weeks)Business Analyst identified 8 participants1 faculty member3 graduate students4 user services librariansServices Working Group and Business Analyst drafted questions

MethodInitial conference call on August 27th, 2007all participantsbusiness analyst systems architectdeveloperQuestions regarding search and findidentifyobtainmanage and useuser workflow

Initial Brainstorming callProsdevelopers present to hear what participants saydevelopers able to prioritize next tasks immediately after the callConsLevel of participant feedback was lower than anticipatedToo much developer involvement?

Participant feedback and development actionNeed to see size and scale of collection to determine how much time to spend searching and browsingImplementation of collection registry and browse collections feature. Need ability to limit a search (once you have results) in many waysImplementation of additional ways to limit after initial searchImplementation of date range limitingImportance of ability to browseImplementation of subject clouds for browsing still working on faceting for additional browsingImportance of visual aids, especially for undergraduate students Implementation of Simile timeline view for resultsThumbnail previews for item selection

Integration with collection registry at UIUC

Adaptively re-used technologyCDL date normalization utilityUIUC thumbgrabber with asset actionsSimile timeline from MITCollectus tool from UVAJust say no to NIH syndrome

Implementation of asset actions using Collectus from UVa

Simile Timeline Integration

Participant wish lists for future developmentLimiting Search Resultsusers want to drill down in the searchcontinue to narrow using different limiters, e.g. rights for useSocial Networking and TaggingImplementation of StumbleIt for full recordsMonitoring Zotero server developmentPenn tagsSorting and NarrowingPlan to implement faceted browsing based on NINES Collex flare development (NIH)

Tell me something I dont know

Lessons learnedAgile Assessment effective in prioritizing tasksMultiple phone interviews and email updates resulted in buy in for participantsBrainstorming session for distributed group not as effective as personal interviewsShorter more frequent 1:1 interviews yielded the best resultsGraduate students were most engaged

Whats nextAdd collectionsWorkflow, including asset actionsSupport for static repositoriesBug fixingTagging? Faceted browseMap rights statement to CC license?

Evaluate ORE model (March meeting)Develop local implementationsSakai @ IndianaFederated search @ UIUCMore SEOAssessment planningSustainability planning

Actively seeking collectionsAdvantage to contributorSEO drives traffic to your collection(s)Benefit from date normalizationSeeking collectionsPertaining to American culture and lifeAll formatsMODScan be mapped from MARCStatic repository setup for contributors without OAI data providers

The DLF Aquifer engineFrom DLF for development to ??? for sustainingConsistent attentionAdministrative supportThe working groupsOrganizational linksCommunity linksResourcesFrom collaborators (directly and through investment in DLF)Grant fundingFees???

Sustaining AquiferPart it outCommunity standardsSoftware and servicesCollectionsTake it to the next level?Affiliate with another (service provider) organizationMove to participant library host

Resources for ongoing supportIf enough people who are viewed as important want it, we will find a way to pay. Paul CourantAssessment during development period is criticalRevenue streamsSubscriptionLibraries pay for servicesIncreased visibility for collectionsMetadata enhancement

Emerging organizations sustaining inititiativesLOCKSS/CLOCKSSFedora CommonsDSpace FoundationEducopia Institute

Questions? Comments?Website: www.dlfaquifer.orgkkott@diglib.org

***Eleven libraries were involved in the Aquifer prototype group. Tennessee joined in early 2005. Cornell and USC joined in early 2006. Yale recently joined.********DLF is a more or less neutral hub. Set up to innovate, elicit consensus from participants on priorities. Demonstrate that collaboration can produce results within a membership organization.What parts are worth keeping for which communities? For the libraries? For the scholars? Who will pay?*