Evaluation of an Information Service: Qualitative Assessment of the

Internet Archive and the Wayback Machine

SIMS Final Project PresentationMay 13, 2002

Pallavi AravindVanessa ArcePeter Roessler

Outline

Introduction to The Internet Archive Introduction to Our Study Research Design and Artifacts Research Findings Recommendations Conclusions

The Internet Archive is building a digital library of Internet sites.

The Wayback machine makes it possible to “surf the web as it was” by accessing web pages stored in the Internet archive.

this “Internet equivalent of the Library of Congress has been capturing and archiving every public Web page since 1996”

The Internet Archive

Introduction to Our Study

We attempt to identify and define ways in which the Archive is serving mankind

Motivation: – its use for documenting the provenance of the

Internet, as “a historical record of cyberspace… [and] as part of an innovative search tool that lets users call up ‘out-of-print’ Web pages.”

– Gilliland-Swetland said, “it is important to understand the societal roles of archives because it is in the fulfillment of these roles that archivists provide the necessary skills and knowledge to contribute to the [current] paradigm.”

  

Research Design and Artifacts

Phase Approach 

Artifact

Initial Assessment

Meet with Brewster Kahle and assess objectives of project.  

Business Proposal

Industry Research

Evaluate use, design, and value of archives.  

Research References

Heuristic Inspection  

Heuristic evaluation and inspection. Heuristic Recommendations

User Testing  

Conduct usability tests with task-oriented operations and questionnaire.

Pre and Post Questionnaire, Task Sheet, Results

Phone Interviews

Formal and anonymous phone interviews with active users of the Archive.  

Interview Questions, Coded Results

Final Analysis  

Analyze data, compile and evaluate results, prepare report.  

Final Report

Research Design and Artifacts

Initial Assessment

Industry Research

Heuristic Inspection

User Testing

PhoneInterview

FinalAnalysis

Research Artifact: Heuristics

Plan– Educated Information Gathering– Evaluate Interface and Usability – Basis for Design of User Tasks and

Questionnaire Methodology

– Independent Evaluator Walkthrough – Comparative Heuristic Analysis

Heuristic ThemesResearch Artifact: Heuristics

Interface Design & Architecture Search Mechanism Search Results User Help

Heuristics

Research Artifact: User Testing

Plan – User Impressions and Interactions– User Learning Curve

Methodology– Performance Driven Tasks – Clear and Direct Design

User Test Themes

Unique Service Interface Design Concerns Information Hierarchy Search Mechanism Limited/Meaningful Search Mechanism Inconsistencies Presentation of Search Results Depth of Information to Search Results IA does not have/provide desired

information

Research Artifact: User Test

User Tests

Research Artifact: Phone Interviews

Plan– Typical Archive User Insight– Verify User Definitions and Characteristics– Gather Opinions of Wayback Machine

Methodology – Administered to Anonymous IA Users– Voluntary Participation – Actual Use Scenarios and Feedback

Phone Interview Themes

User ProfileFirst ExposureUses Satisfaction and RecommendationsValue and Vision

Research Artifact: Phone Interviews

Phone Interviews

High Level Themes

Vision of Wayback vs. User Practicality Look and Feel Search & Results Technical Capability Communication User Threshold, Expectations and Use

Scenarios

Final Analysis

Look & Feel

IntuitiveSearch Interface

InvaluableResource

What?Why?

Search & Results

Methods to access the archive.

Provision of advanced search

How is the information stored?How is the information retrieved?Where is the boundary between archive and real web site?

Frequency of ArchivalRequest a crawlPolicy on IP

Communication

FAQHelp!

InternetArchiveArchival

Retriev

al

Internal

Storage

Conclusions

Aesthetic and Minimalist Index Page Design

Ability to Make Personalized Collections Provision of a Directory Service URL Recommender System Advanced Faceted Search

– Time and Event Based Faster Server Response Time Standard Frequency of Archival Ubiquitous Documentation

Thank You & Audience Questions

Tremendous thanks to: Our Project Advisor, Professor Peter Lyman; Internet Archive Founder and Director, Brewster Kahle; all SIMS classmates who participated in our research, and Internet Archive users everywhere.

Project Website located at:

http://dream.berkeley.edu/wayback/index.htm

Q & A ?