barriers to interoperability technical and not so technical william e. moen school of library and...

Barriers to InteroperabilityTechnical and Not So Technical

William E. Moen<[email protected]>

School of Library and Information SciencesTexas Center for Digital Knowledge

University of North TexasDenton, TX 72603

5th Annual GILS Conference, April 7-10, 2003 Lisle, IL

Moen 5th Annual GILS Conference -- April 8, 2003 -- Lisle, IL 2

Interoperability: Volume or value

“…information production is growing at about 50% a year…

Yet the amount of time people spend consuming is growing only 1.7% a year…

As digital production and storage grow ever cheaper…a critical task ahead will be to stop volume from simply overwhelming value.”

John Seely Brown and Paul Duguid, The Social Life of Information, 2002


Defining Interoperability

System-oriented definition The ability of two or more systems or components to

exchange information and use the exchanged information without special effort on either system

User-oriented definition User’s ability to successfully search and retrieve, or

otherwise interact with information in a meaningful way and have confidence in the results

The condition achieved when two or more technical systems can exchange information directly in a way that is satisfactory to users of the systems (AAP)


Assessing Interoperability

Binary Interoperable Not interoperable

Continuum More or less interoperable Acceptable levels of interoperability


Fundamental assumptions Your users will interact with resources other than

those available locally Your system will interact with other systems’

content Other systems will interact with your content Systems and content were often meant for local or

proprietary use

In the networked environmentdiverse systems and varied content

must interact to serve users


The foundation: Communication protocols Interoperability is about communication – albeit

between machines/software Two systems interchange bits/bytes Protocols (computer-to-computer communications)

govern the exchange of bits/bytes Rules governing the structure, order, and content of the

bits/bytes/messages between machines


Protocols and record formats/markup Protocols enable communication between systems Record formats/syntax/markup address resources HTML = HyperText Markup Language

Markup language for simple web documents Display oriented

MARC = Machine Readable Cataloging Communications format for exchange of bibliographic

information XML = Extensible Markup Language

Markup language for web environment Content oriented


Factors affecting interoperability Multiple protocols

Z39.50, HTTP, SOAP, etc.

Multiple data formats, syntax, metadata schemes MARC 21, UNIMARC, XML, / ISBD/AACR2-based, Dublin Core

Multiple vocabularies, ontologies, disciplines LCSH, MESH, AAT

Multiple languages, Multiple character sets Multiple and disparate systems

Information retrieval systems, search functionality, etc.

Indexing, word normalization, and word extraction policies


Assumptions

Increasing interoperability factors

Increased complexity/costs to achieve interoperability

How to identify barriers to interoperability: In general? In specific applications?


Interoperability TestingRealizing the Vision of Networked Access to Library Resources: An Applied Research and Demonstration Project to Establish and Operate a Z39.50 Interoperability Testbed

A U.S. Federal Institute of Museum and Library Services National Leadership Grant

Research and demonstration project to: Establish a rigorous and trusted Z39.50 testbed Develop methodology and metrics for assessment From demonstration operational testbed

Goal: Improve Z39.50 semantic interoperability among libraries for information access and resource sharing

FOR MORE INFORMATION, VISIT THE PROJECT WEBSITE…

http://www.unt.edu/zinterop/


Components of the Testbed Test dataset

400,000 MARC 21 records from OCLC Z39.50 reference implementations

Z-client, Z-server, information retrieval system Test scenarios & searches

Searches with known result records from dataset Benchmarks

Results of test searches against reference implementations


Interoperability findings Successes:

Z39.50 profiles improve interoperability Z-clients and Z-servers can be configured to improve Common indexing policies support better interoperability

New problems: Word extraction (what constitutes a word) Data normalization (special characters, diacritics) Indexing (leading articles, “first in field” searches)

What is good enough interoperability?


ZLOT project The Library of Texas

A statewide virtual library to extend access to information

Z Implementation Component of the Library of Texas Developing a standards-based approach for the virtual library Identifying requirements for resource discovery across library

catalogs and online databases Demonstrating feasibility and utility of resource discovery tools

• Collection Boundaries

• Organizational Boundaries

• Format Boundaries

FOR MORE INFORMATION, VISIT THE PROJECT WEBSITE…

http://www.unt.edu/zlot/


One approach for framing interoperability Information communities / communities of practice Networked information retrieval occurs within and

across information communities Information communities

Focal community Extended community Extra community

Costs to achieve interoperability vary depending… Within focal community Within extended community Between disparate communities


Focal Community Community agreements exist (e.g., standards and rules) Interoperability factors reduced…more easily achieved

Libraries as Focal Community

Relatively homogeneous Standards-based MARC records Content and structure prescribed by AACR Commonly available access points Use of controlled vocabularies Z39.50 widely implemented But…


Information exchange within/between…

Focal Community(e.g., Libraries)

Focal Community(e.g., Archives)

Focal Community(e.g., Museum)

Extended Community(e.g., Cultural Heritage)

Focal Community(e.g., Geospatial )

Extra Community

Focal Community(e.g., Natural History

Museums)

Extended Community

Focal Community(e.g., Geospatial )


GILS as information community Conceptualizing GILS as information community Way to discuss barriers to interoperability:

Types of data and resources? Common resource description scheme Commonly used database semantics? Common record syntax (e.g., XML) Commonly understood access points/search concepts? Commonly used controlled vocabulary? Common protocols for information retrieval and

exchange?


Services context for interoperability Information communities provide setting Information communities offer services

Value-added services to end users Infrastructure services for providers

Mapping a service environment … Needs Services Service interfaces Content

… to identify interoperability barriers/challenges


Characterizing the networked library Virtual library components An environment for provision of

Services Resources

Not a digital library Not all resources available are in digital/electronic form

An evolving product that responds to Users’ needs Available resources Current and emerging technologies


Service-centric networked library

UserGroups

Resources

Technology

Management

SERVICES


Networked library components User groups

Local, onsite Distributed, remote

Services Local Collaborative Remote

Resources Information

• Local• Distributed

Human

Technology Local applications Distributed applications Standards

Management Local Collaborative


User-driven services

LIBRARY SERVICES

UserGroup

UserGroup

UserGroup

UserGroup

LibraryResources

LibraryTechnology


Building services for the networked library What are the requirements for the services? What are alternatives for deploying the services? What technologies are available to support the

services? What standards are available? What are the tradeoffs of standards and non-

standards approaches?


Services for the networked library Infrastructure/internal

Collection identification, selection, and acquisition/access

Collection building and digitization

Collection organization and preparation

Inter-organization access Trust and authentication Billing and payment Preservation and archiving

Patron Resource discovery service Resource access service Reference service Instruction service Patron account service


Virtual library services environment


Architecture

PROVIDER LAYER

SERVICES LAYER

USER LAYER

Communication Protocols

Communication Protocols


Provider layer

DigitalContent

Librarian Library Supplier

Provider Type Dependent on VL Service

ResourceDiscovery

Service

Virtual Reference

Service

ResourceSharingService

DocumentDeliveryService

USER LAYER1

3

2


Services layer

Standard Information Interchange Among Services

Financial

Authorization

ResourceSharing

PatronAccount

LibraryInformation

EducationResourceDiscovery

Reference Personalization


Resource discovery: Services view

Browse

SelectCollection

Evaluate

Search

Common User Interface

Personalize

Save

Acquire


Resource discovery: Architectural view

CONTENT TexShare TRAIL VC Others

USER LAYER Library Home Office Mobile

SERVICES LAYER Resource Discovery

Z39.50

HTTP

OTHERDirectory

Authentication

SOAP

SOAP


Finding / exposing Finding

I can find only what’s been exposed Exposing

How content is exposed determines the how and if I can find it Common agreements on exposing and finding

Protocols Syntaxes Semantics

• Database• Language

Goal is to connect users to information Strategy is interoperability on multiple levels (protocol,

syntax, semantics, organization, etc.)


Service interfaces framework


Resource discovery: Operational view

Content

Content

Content

Content

Content

OAIRepository

Standards-basedSearch & Retrieval

Protocols

Proprietary or Non-standard

Search & Retrieval Protocols

Content

Content

Content

Gateway

CommonInterface


Infrastructure standards -- NISO ANSI/NISO Z39.50: Information Retrieval (Z39.50): Application

Service Definition and Protocol Specification ISO 10160 & ISO 10161: Interlibrary Loan Application Service

Definition & Protocol Specification ANSI/NISO Z39.83: Circulation Interchange Protocol (NCIP) ANSI/NISO Z39.84: Syntax for the Digital Object Identifier - 2000 ANSI/NISO Z39.85: Dublin Core Metadata Element Set - 2001 ANSI/NISO Z39.88: The OpenURL Framework for Context-Sensitive

Services (2002) ANSI/NISO Z39.89:The U.S. National Z39.50 Profile for Library

Applications (2002) Networked Reference Services (standards development underway) MetaSearch Initiative (may lead to new standards activities)


Standards supporting services Each standard enables different services

Networked information retrieval (Z39.50) Automated interlibrary loan (ILL) Circulation applications (NCIP) Reference linking (OpenURL) Resource description (Dublin Core)

But they can work together to support robust and interoperable information access and resource sharing services


Web services

Building Blocks for Constructing Applications

Service A

Resource Discovery

Service B

Authorization


Web services: Finding a service

Service A

Resource Discovery

Service B

Authorization

ServiceRegistry

ServiceRequestor

ServiceProvider


Web services: Enabling technologies

ServiceRegistry

ServiceRequestor

ServiceProvider

XML-Based Information Exchange

SOAP

UDDI - WSDLUDDI - WSDL

WSDL

UDDI: Universal Description, Discovery and Integration WSDL: Web Service Description FrameworkSOAP: Simple Object Access ProtocolXML: Extensible Markup Language


Web services

ResourceDiscoveryService

Service B

Authorization

ServiceRegistry

UDDI - WSDLPublishUDDI - WSDL

Find

WSDL - SOAP

Invoke Service


ZING – Z39.50 International Next Generation Make intellectual/semantic content of Z39.50 more

broadly available Make Z39.50 more attractive by lowering barriers to

implementation Use of XML – to represent and encode data Use of HTTP – for transport Use of SOAP – for interaction between client and server

based on Remote Procedure Call (RPC) Several initiatives: ZOOM, ez39.50, ZeeRex,

SRW/UFOR MORE INFORMATION, VISIT THE PROJECT WEBSITE…

http://www.loc.gov/z3950/agency/zing/zing.html


Search/Retrieve Web Service – Overview Builds on Z39.50 and web technologies Retains Z39.50 concepts:

Results sets Abstract access points Abstract record schemas Explain & Diagnostics

Web technologies: XML, SOAP/RPC, HTTP Combines several Z39.50 features into two “operation types”

Search/Retrieve Explain

Provides a standards-based method for representing and issuing queries, and retrieving records


Interoperability and the user Technical interoperability as foundation Value-adding services respond to user tasks Usability of the service

Reliable Predictable Ease of use User interface …


Final thoughts Interoperability is essential Interoperability is multi-faceted Interoperability is ultimately based on agreements

Formal standards Community agreements

Agreements result from Social processes Willingness to collaborate Balance of local needs in the networked environment

It’s the “not so technical” we need to work on

barriers to interoperability technical and not so technical william e. moen school of library and...

Documents