HARVARDE-JOURNAL ARCHIVING

STUDY

Dale FleckerJune, 2002

JOURNAL ARCHIVING IN THE PAPER ERA

• Large-scale redundancy• Access copy and archival copy usually the

same• Not just storage, but preservation

– includes environmental control, library binding, repair, reformatting. . .

• Deliberate, long-term archiving largely the role of national and research libraries

E-JOURNAL MODEL IS DIFFERENT

• “Copies” are remote, held in publisher systems– Not replicated across different institutions

• Perpetual license provides limited comfort in the absence of independent copies

• Long-term preservation involves very different issues than day-to-day access

E-JOURNAL ARCHIVING A GROWING PROBLEM

• Libraries bearing double costs– the e-journals users prefer– the paper for preservation

• Publishers cannot convert totally to digital– authors and editors distrust e-only journals because of

concerns about persistence– libraries demand paper for preservation

• Libraries preserving paper version, but electronic more complete, increasingly the copy of record

MELLON E-JOURNAL ARCHIVING PROGRAM

• 13 institutions invited to submit proposals for a planning projects

• Two approaches – Large-scale distributed replication (LOCKSS)– Centralized archives serving a wider

community

CENTRAL ARCHIVES PLANNING PROJECTS

• Publisher-based – Harvard (Wiley, Blackwell, University of Chicago

Press)– Penn (Oxford and Cambridge University Presses) – Yale (Elsevier)

• Discipline-based – Cornell (agriculture), – NYPL (performing arts)

• Dynamic e-journals – MIT

FOUR BASIC ASSUMPTIONS

• Archive should be independent of publishers– responsibility of institutions for whom archiving is a

core mission• Archiving requires active publisher partnership• Address long timeframes (100 years?)• Archive design based on Open Archival

Information System (OAIS) model

CENTRAL ARCHIVE MODEL

• Archive negotiates relationship with publisher• Publisher deposits content regularly• Content accompanied by metadata to support

discovery and preservation • Archived content only accessible under specific

conditions• Archive assumes responsibility for long-term

preservation

SOME INTERESTING QUESTIONS

• What is archived?• In what format?• When is archive accessible?• Who can access archived content?• What does the archive “preserve”?• Who does archiving?• How is the archive paid for?• How is the archive governed?

WHAT CONTENT IS ARCHIVED?

E-journals not simply articles….

SOME COMMON STUFF

• Journal description• Editorial board• Instructions to authors• Rights and usage terms• Copyright statement• Ordering information• Reprint information• Indexes

• Career information• News• Events lists• Discussion fora• Editorials• Errata• Reviewers• Conference

announcements

HARD AREAS

• Masthead, “front matter” stored as web pages, not in content management systems

• No control over the format of “associated materials” (datasets, images, tables, etc.)

• Advertising very complex– dynamic, frequently from third party, can involve

country-specific complexities• Links frequently separate from articles

– regularly updated, sometimes dynamic

OUR INCLINATION

• Exclude little except advertisements – based on discussions with librarians and scholars– different from most “local loading”

• Articles include supplementary materials• Include an “issue object” in addition to the

article components– masthead, news, jobs, meetings, etc

Format for archived articles?

PDF?

• PDF almost universally available from publishers– and the only format available for some journals

• There are qualms...– proprietary– marked-up for display, not meaning– supports limited functionality– long-term “preservability” unclear– unlikely to remain the universal format over time

MARKED-UP TEXT?

• SGML/XML increasingly common– and likely to become more so

• Greater functionality, easier migration as technology changes

• Complex– DTDs vary widely from publisher to publisher– DTDs far from stable– archive documentation and rendering would be complex

“INTERCHANGE” ARTICLE DTD

• Intended for exchanging content between independent players

• Reduces complexity of interaction – archive needs to document, migrate, and display only

one format• archive can choose whether to maintain articles in

interchange DTD, or transform at ingest for long-term storage– publisher needs deposit only one format for all archives

“INTERCHANGE” ARTICLE DTD

• Mellon, Harvard, National Library of Medicine, 2 consultants (Inera, Mulberry) working on draft standard DTD

• Design based on current publisher practice– must be easy for publishers to produce– homogenizes many elements – leaves options in some difficult areas– eliminates elements specific to individual publisher

delivery systems

INTERCHANGE DTD ISSUES

• How low is the common denominator? • What gets lost?

– inevitably sacrifices some functionality and original appearance

• Transformation from publisher’s “native” DTD involves risks

• Some technically difficult areas– extended character sets, mathematical and chemical

formulae, tables. “generated text”

SGML/XML QUALITY CONTROL PROBLEM

• SGML/XML is an output rather than the input for many publishers today– may not fully reflect the output (PDF, print) that

users see day-to-day…how do you know it is good?• If SGML/XML is transformed for deposit, errors

can introduced• Quality control of ingested content is expensive

but critical for a sound archive

ARCHIVE MORE THAN ONE FORMAT?

• Publisher-based archive must accept PDF in any case (only format available for some titles)– so include both SGML and PDF when available?

• belt and suspenders

• Accept publisher’s original SGML also?– preserve information lost in conversion to

interchange DTD– maintenance over time problematic

WHEN IS ARCHIVE ACCESSIBLE?

• Most publishers instinctively prefer “dark” archives– does not compete with publisher’s service

• If “dark”, what “trigger events” make it accessible?– after a given period of time (‘moving wall”)?– when content is not otherwise accessible (“failsafe”)? – only when content enters the public domain?

IS “DARK” DANGEROUS?

If content is dark, how do you know it is still good?

(real users are the best auditors)

WHO CAN ACCESS ARCHIVE CONTENT?

• Just other subscribing institutions?– does the archive need to maintain complex records of

license rights?• defining licensees a nightmare• tracking license changes over time another nightmare

• Individual subscribers? – an even greater nightmare

• Everybody?– dramatically easier to administer

WHAT DOES THE ARCHIVE PRESERVE?

• Preservation is a format-by-format issue– and most e-journals are composed of many

formats• How much “look and feel” preserved?• Just preserve the “core intellectual content”?• Does archive insure content remains

“render-able” as technology changes?

HARVARD’S DIGITAL REPOSITORY

• Repository specifies preferred (“normative”) formats, which will be kept useable

• Just maintain bits for others– for e-journals this is likely for many “associated

materials” (datasets, models, etc.) • generally accepted in ANY format• maintaining the viability of such wildly heterogeneous

materials unrealistic– keep unaltered for future “digital archeology”

WHO DOES ARCHIVING?

• “Common good” activity– model based on a few archives serving many

subscribers• Is this an appropriate role for individual

universities?– research libraries have technical capability,

relationships with publishers and subscribers– BUT how archiving would be paid for is central…...

HOW IS THE ARCHIVE PAID FOR?

• First question: who benefits?– publishers, libraries, authors, scholarly societies…– is there a way to share costs?

• Cost categories include– preparation of “archivable” objects– ingestion and quality control– long-term storage– preservation

PROPOSED MODEL

• Publisher assumes cost of preparing objects in standard format (whenever possible)

• Deposited material accompanied by two part fee from publisher– ingest fee to cover up-front costs

• varies with publisher effort to create easily archived objects???– “dowry” to create maintenance endowment

• Real funding sources include subscribers, authors, societies

HOW IS THE ARCHIVE GOVERNED?

* Publishers hand their its intellectual property to independent party -- do they have a continuing say?

* Are there other stakeholders whoshould also have a say?

HARVARD’S MODEL ARCHIVE

• Accept content for all titles a publisher produces– archive as many journal elements as possible

• Maintain an archive serving the entire community

• Store and maintain more robust formats (e. g., XML) when possible

• Collect metadata to support administration and preservation

HARVARD’S MODEL ARCHIVE

• Requires only a few archival copies of any given journal

• Archive assumes responsibility for preservation migration when canonical versions deposited

• Organizational and economic model difficult

NEXT?

Over to Kevin….