archiving websites containing archiving young composer...

5/16/17

1

ArchivingWebsitesContainingStreamingMedia

HowardBesser,NYUh5p://besser.tsoa.nyu.edu/howard/Talks/

Besser-IS&TArchiving16/5/2017 1


•  Backgroundissuesandproblems•  ArchivingYoungComposerwebsites

– OurTechnicalCollaboraOon– OurCollaboraOonwithContentCreators(preservaOon,accesscontrol,rights,agreements)

– Workflows– Howthingswilllook– EvaluaOon

•  ImpactbeyondthisProject


BACKGROUNDISSUESANDPROBLEMS


WebArchivingposeschallenges

•  Anygivenwebpagemaybeupdatedfrequently

•  Weblinksconstantlybreak(404errors)•  Fewtools/servicesexistfor“Curated”webarchiving(Archive-It,CDL’sWAS),andtheyrequiresignificanttraining/experiencetolearn,butwedohaveint’l-acceptedformat(WARC)


ManyparametersneedtobesetforWebArchiving

•  Frequencyofcrawls•  Depthofcrawls(#ofhops)•  StarOngpointsofcrawls(seeds)


Otherissuesfordevelopinggoodcrawls

•  Qualitycontrol/assurance•  Workflows•  Fidelitytooriginalwebpages•  Howenduserwillnavigateandviewit


5/16/17

2

Archive-It

•  TheleadingapplicaOon/serviceforcuratedwebarchivinginNorthAmerica

•  RunbytheInternetArchive,andismuchmoretargetedandcuratedthantheirWayBackMachine

•  IsbasedonCrawlersogwaredevelopedbyIA(Heritrix)in2003-2004

•  IsverypooratcapturingstreamingaudioorvideoaswellasinserOngitproperlyintoacomposedwebpage-


Archive-ItIssuesw/StreamingMedia






Archive-Itscreenshotsgeneratedaspartofourproject-

•  ByLorenaRamirez-Løpez


Archive-ItIssuesw/StreamingMediaFireFoxversion39.0.ScreenshotofTarikO’Regan’ssitetaken2015/10/05


5/16/17

3





Archive-ItIssuesw/StreamingMediaFireFoxversion39.0.ScreenshotofTedHearne’swebsitetaken2015/10/05


Somesourcesofstreamingissues

•  Problemswithcapturingresourcesresidingon3rdpartyservices(YouTube,Vimeo,Soundcloud)

•  ProblemswithhowfaithfullytheA/VmaterialsarecapturedandplacedbyArchive-It

•  ProblemswithwebsitesgeneratedthroughsitebuildingplamormssuchasSquarespace


ARCHIVINGYOUNGCOMPOSERWEBSITES


ArchivingComposerWebsitesh5p://www.nyu.edu/about/news-publicaOons/news/2015/03/27/nyu-libraries-to-team-with-internet-archive-to-preserve-high-

quality-musical-content-on-the-web.html

•  Collect,preserve,&makeavailableWebsitesofComposers

•  $480,000grantfromMellonin2015toNYULibrary/MIAP/InternetArchive

•  Dealingwiththeissuethatcontemporarycomposerwebsitesgoupanddown(andalsoincorporaterelaOonship-buildingbtwncomposerandfans)

•  AddressingtheproblemsofcollecOngstreamingmedia•  AlsoselecOvelycollecOnghigh-qualityversionsthatareusedtogeneratethestreams,andallowingfutureresearcherstosee/hearthehigherqualityversions


5/16/17

4

ArchivingComposerWebsites


•  DevelopgoodandongoingrelaOonshipsbtwnLibrariesandComposers

•  DevelopTrust–  fordevelopingcollecOons,andconOnuingtoaddtothem–  forPolicyreasons

•  Examinewhattypeoferrorstakeplace–  howfaithfullyaudiovisualmaterialsarebeingcaptured–  howresourcesthatresideonthird-partyweb-services(YouTube,Vimeo,Soundcloud)are(not)displayedwithinArchive-It’sinterface

–  IssueswwebsitesgeneratedthroughsitebuildingplamormssuchasSquarespace

•  Findwaystofixthoseerrors

MetricsAccomplished(asofJan2017)

•  172Composersitescrawled,scoped,assessedforquality,&analyzedforproblems(feedingintoIAdevelopmentwork)

•  800QA/QCreportsgenerated•  IniOalwebarchivingagreementfrom165Composers(25fromNPR’s100)

•  IdenOfiedwebsiteinfrastructuresencounteredandcreatedaclassificaOonmatrix-


WebsiteInfrastructureencountered


ProjectTeam•  JeffersonBailey(InternetArchive)•  HowardBesser(MIAP)•  LoriDonovan(InternetArchive)•  AprilHathcock(Lib/ScholComm)•  NicoleGreenhouse(Lib/ACM)•  CarolKassel(Lib/DLTS)•  Sco5Statland(MIAP)•  DonaldMennerich(Lib/ACM/DLTS)•  DavidMillman(Lib/DLTS)•  CourtneyMumma(InternetArchive)•  RobinPreiss(Lib/AFC)•  LorenaRamirez(MIAP)---specialthanks!•  MichaelStoller(Lib/C&RS)•  KentUnderwood(Lib/AFC)•  ChelaSco5Weber(Lib/AFC)


OURTECHNICALCOLLABORATION:CRAWLING


NYU/IACollaboraOon


5/16/17

5

NYU/IACollaboraOon


TradiOonalCrawlers


•  Archive-ItandotherwebarchivesuseHeritrix•  Followlinks,capturemostwebcontent•  Lesssuccessfulwithstreamingvideoanddynamiccontentexecutedinthebrowser

•  Umbrahelps

BROZZLER!

“browser” | “crawler” = BROZZLER

Logo: Noah Levitt Besser-IS&TArchiving16/5/2017 27 Besser-IS&TArchiving16/5/2017 28

BrozzlerSystemArchitecturev1


OURCOLLABORATIONWITHCONTENTCREATORS


5/16/17

6

YoungComposersCorpus

•  BeganwithNPR’s2011listof“100ComposersUnder40”

•  91of100haveownself-containedsites•  Asof5/2016havewri5enagreementswith165Composers(25ofthemfromNPR’slist)

•  Willrecruit10ofthemforenhancedarchiving(uncompressed;be5erthanwhatisonwebsite)–  Thiswillrequireanaddedappendixtocontract/agreement(whichmayinvolvedarkarchivingand/orrestrictedaccess)


BuildingrelaOonshipswithComposers

•  EngagethemwiththeideaofpreservingtheirWebsite

•  Aretheywillingtogiveusricherversionsofcontentontheirsite?

•  Aretheywillingtomakeall(orjustpart)ofthecontentfreelyaccessible?Dotheywanttoembargosomecontentinadarkarchive?

•  DonorAgreement/Contract-


DonorAgreement/Contract

•  Havebeenworkingonthiswithlawyersforapproximatelyoneyear

•  Havehadfairlystablelanguageinitfor6months,and2contractsalreadysignedandreturned

•  Doesdefaulttoallowinguscompleterightsforreformaungandforallowingresearcherstosee/hearallhighqualityversionsatminimumon-site– AndthusfarallComposerscontactedhaveagreedtothoseprinciples(butnotnecessarilytothecontractuallanguage)


LongTermPreservaOonforScholarship


Highestquality;futurelibraryprocesses


ComposerindicatesrestricOonsonAccess


5/16/17

7

WhoarecerOfiedusers?


ARCHITECTURE&WORKFLOWS


Architecture&Workflows

•  Fullcopyofallwebsites(inclrichercontent)storedinNYURepositoryandaccessiblethroughNYUFindingAids

•  MetadataisinArchiveSpace•  ConnecOonsbuiltoffofArchiveSpaceback-endAPI


CurrentDevelopmentwork

•  Supplyingaseparateaudioplayer?•  HiringaDigitalArchivist•  PreciseformsofnavigaOonbtwnArchiveSpace,Archive-It,andrichercontentwithinNYU’sdigitalrepository

•  API-


API•  WhatIAneedsfromNYUAPI

–  APIURL–  CredenOals(username,password)->AuthenOcaOonToken()–  RepositoryID–  ResourceID

•  WhatIAwillreturnasJSONarray–  UnitTitle–  Creator–  DataExpression–  ExtentStatement–  TechCharacterisOcs–  [SomethingBasedonAccessRestricOon,i.e.canitbestreamed]???

•  WeSpeakEtruscan,1993May21,23.5MB,1AIFFfileStereouncompressed16bit/44.1K

•  TheDreamofInnocenceIII,1998March26,150MB,1AIFFfileStereouncompressed16bit/44.1K


HOWTHINGSMAYLOOK


5/16/17

8

UserQueries

•  UserbrowsesthroughArchive-It•  UserseesthatA/Vcontentexists(andinsomecases,itwillincluderichercontent,butsomeofthatmightbeaccess-restricted)

•  Archive-IthandsoffusertoNYU(eitherdirectlytoA/Vcontent,ortoFindingAid)


OneopOonforQueries


OneopOonforhighqualitycontent

•  OnarchivedwebsitepagelisOngcomposer’scontent,userseesamessagethathigherqualitycontentisavailable,with:– AccessrestricOons,ifapplicable– Linktorelevantfindingaid–  (lookinglikefollowingimage)-

Besser-IS&TArchiving16/5/2017 45 Besser-IS&TArchiving16/5/2017 46

EVALUATION


EvaluaOonforImprovement

•  ComposersandtheirsaOsfacOonwiththewaysinwhichaudienceswillbeabletoviewarchivesoftheirwebsites

•  Researchers,andwhetherthecontentandfuncOonalityofthesewebarchivesworksforthem

•  Tweakingwhatwedoinordertobe5erserveCreatorsandResearchers


5/16/17

9

ScheduleandMethodologyforEvaluaOon

•  Dec2017—Scheduleone-on-oneinterviewswithsetsofcomposersandResearchers

•  Jan-Mar2018—Onehourindividualsessionswith10Composersandalsowith10Researchers,havingthemlookattheuserinterfaceandconductqueries–  Composers:AretheysaOsfiedwithhowaudienceswillbeabletoviewthe

archivalcopiesoftheirwebsites?Isitbe5erorworsethantheirownlivesites?AretheysaOsfiedwiththeaudioandvideoplacementandquality(aswellasopOons)?AretheycontentwiththeDonorAgreement?Whatchanges/improvementsmightbemadetoanyofthese?

–  Researchers:Cantheyfindwhattheyneedinthewebarchive?Isitdifficult(clunky)touse?Whatpartsdon’tworkwelloraren’tintuiOve?WewanttoidenOfywhatchangesinthecontent,funcOonality,ornavigaOonfeatureswouldimprovetheiruserexperience

•  Apr-May2018—ConstrucOonofEvaluaOonSummarycontainingthelistofimprovements/changesthatshouldbemadetotheArchivingproject

•  June-Aug2018—Implementthechanges


IMPACTBEYONDTHISPROJECT


ImpactBeyondthisProject•  Archive-Itwillbeabletobe5erhandlestreamingmedia,anddisplayitinpropercontext

•  WewillhavearchitecturesandworkflowsforArchive-Ittointeractwithricherlocalresources(aswellasexamplesofhowinteracOonandnavigaOoncanproceedbtwnArchive-It,ArchiveSpace,FindingAids,andaninternaldigitalrepository)

•  ModelsforinteracOonbtwncreatorsandcollecOngorganizaOonswillhavebeendeveloped(incldonoragreements)

•  Wewillhavepreserved100+++websitesofyoungcomposers



•  h5p://besser.tsoa.nyu.edu/howard/Talks/•  h5p://www.nyu.edu/about/news-publicaOons/news/

2015/03/27/nyu-libraries-to-team-with-internet-archive-to-preserve-high-quality-musical-content-on-the-web.html


archiving websites containing archiving young composer...

Documents