organizational usability of digital libraries in the...

Proceedings of the 29th Annual Hawaii International Conference on System Sciences - 1996

Organizational Usability of Digital Libraries in the Courts

Margaret Elliott and Rob Kling

Department of Information and Computer Science and Center for Research on Information Technology in Organizations, University of California, Irvine, CA 92717 US

(kling, melliott) @its. uci. edu

Abstract

Digital libraries (DLs) are the focus of professional movements in the medical, legal, and scholarly communities. DLs are being used increasingly nationwide in the legal profession for research; computer-integrated court reporting; and for trial presentations of multimedia documents and evidence photos. In this paper, we present results of a study which contributes to DL research by examining some new DL design concepts within the legal domain. Two key forms of DL usability - interface and organizational - are discussed. While the Human- Computer-Interaction (HCI) research community has made progress in improving interface usability, organizational usability is less well understood. We introduce the concept of organizational usability which refers to the mix between a computer system’s design and an organization’s characteristics such that the system can be effectively integrated into the work practices of members of the organization and is socially accepted by them. We explore the technical and sociotechnical dimensions of organizational usability by applying them to DLs used in the California Superior Courts.

Introduction

DLs are the focus of professional movements in the medical [ll, legal [2, 31 and scholarly communities [4]. In this paper, we present results of a study which contributes to DL research by examining some new DL design concepts within the legal domain. DLs mean different things to various researchers. DLs, for this paper, are defined as collections of digital documents used as dynamic or archival repositories of information in the form of text, video, graphical pictures, or sound. Increasingly, nationwide there has been an influx of DL usage into the legal profession for research; for computer-integrated court reporting; and for trial presentations of multimedia documents and evidence photos [5]. In this paper, legal research DLs are defined as electronic corpuses containing text of statutes; case law; and legal references to up-to-date rulings on cases and statutes.

The purpose of this paper is to present results of a study on the use of legal research DLs in the California Courts. The goal of the study is to examine what characterizes an effective organizational usability design of a DL, what institutional and occupational cultural factors influence usage patterns, and how organizational usabiltiy can improve organizational effectiveness. We chose the legal profession as exemplary of DL usage because research is an integral part of their job and is vital to their work. For example, during a trial, lawyers and judges need to know the applicable law related to a specific case. DL legal research expedites document and oral argument preparation with faster and sometimes, more comprehensive material than its manual counterpart. In order for these systems to be integrated into current work practices of attorneys and judges, they have to be designed for ease of use and learning, and they need to be socially accepted by the organization.

For example, consider this situation involving a public defender (PD) who was unable to access a legal research CD-ROM during an interview. When the PD tried to “login” to the networked CD-ROM, entry was blocked because Ihe software administrator had changed the required password without prior warning. To obtain this password, the user must phone the administrator during regular working hours. Since the interview took place after hours at 600 p.m., this was problematic. The administrator is not always readily available during normal working hours and this lockout condition occurs frequently. In fact, this PD keeps a personal DL in WordPerfect of pertinent legal issues which she updates frequently so as not to rely on the CD-ROM for digital documents. For this person, the DL was not usable.

This is an example of how computer systems can be designed for usability at an interface levei but not match the organizational requirements for effective use. The basic concept of usability is that “any system designed for people to use should be easy to leam (and remember), useful, that is contain functions people really need in their work, and be easy and pleasant to use”[6, p. 3001.

We introduce the concept of organizational usability which refers to the mix between a computer system’s design and an organization’s characteristics such that the

62 1060-3425/96 $5.00 0 1996 IEEE

Proceedings of the 1996 Hawaii International Conference on System Sciences (HICSS-29) 1060-3425/96 $10.00 © 1996 IEEE

system can be effectively integrated into the work practices of members of the organization and is socially accepted by them. In [7], two key forms of DL usability - interface and organizational - are discussed. We extend our research by adding explicit dimensions, separating the dimensions into technical and sociotechnical categories, and examining them with detailed empirical data. These dimensions are derived from our own experiences with previous studies, with personal DL usage, and from empirical data collected in the courts. We use them to assess the organizational usability of legal research DLs used by attorneys and judges in the California Superior Courts.

The HCI research community has made progress in improving interface usability [6,8]. Less progress has been made in the HCI community in devising methods for the sociotechnical approach which promotes the appropriate match between technical and social systems [9]. This approach has not been widely adopted and has proven difficult to use by designers because no specific technical design directions have been given.

In the management information systems (MIS) field, Markus and Robey [lo] defined the concept of organizational validity as the “fit” between an information systems and its organizational context of use. This goes beyond technical validity to include users’ motivations or cognitive use, structural dimensions of the organization, the distribution of power in the organization, and the interface between the organization and its environment. The utility of organizational validity lies in its descriptive value, not normative application.

There are significant differences between MIS and DLs. MIS are a set of interrelated components that collect, process, store and distribute information to support managers in the functions of planning, controlling, and decision making. DLs are repositories of information with the purpose of providing individuals with information retrieval and archival functions. DLs may replicate the content of books or they may be a collection of text, video, and sound corpuses. Unlike MIS, which provide managers with specific reports to oversee group operations, DLs are used at an individual level for a specific research task involving information storage and retrieval. We use organizational usability dimensions to assess these unique features of DLs at both the technical and sociotechnical levels of system design

Designers and configurers of DLs tend to focus on the content of the corpus and less on the dynamics of everyday use. For example, in [ll], DLs were viewed as being highly usable if two conditions were met: 1) the system contained appropriate content and 2) the system had a user-friendly interface design utilizing the most effective HCI interface-building techniques.


We challenge this view with empirical data showing that a DL can have a valuable corpus and a superior interface but some will find it unsuitable for various reasons - at both a personal and organizational level. By contrasting organizational usability dimensions across many cases, we show varying degrees of the “fit” of the same DL to various organizations. We maintain that organizational usability is a vital concept for effective system usage, and strive to inspire people to consider it when designing or configuring DLs for an organization.

Table 1. Legal professionals interviewed

Research method The research design centers on a qualitative case study

[12]. In progress since January 1995, this ongoing study has included field interviews; participant observation: study of documents and computer manuals; and hands-on use of legal research DLs. Table 1 shows the number of attorneys, judges and support staff who have been interviewed. All interviews were conducted as semi- structured with a predetermined set of questions; however, often the interviewee provided insights not directly related to the questions. Appendix A includes a summary of these questions. Most interviews were conducted face-to-face with a few completed on the phone.

The research plan involves the study of three courtrooms in the same county (County X) within the California Superior Criminal Courts System with varying degrees of computer usage: Court A with low use of technology (law research); and Court B with average use (law research, real-time court reporting) and Court C with high use (law research, real-time court reporting, multimedia evidence preparation). These varied levels of computer usage were selected to determine whether increased computer usage in general would have a positive influence on DL usage. To date, we have investigated Court A, partially completed Court B, and will complete Court C in the future. Interviews were conducted with DAs, PDs, judges and one criminal defense attorney. In addition, we continue to interview PDs, DAs, and civil attorneys. See [13] for the study of civil attorneys and their comparison to criminal.

The dependent variable in this study is the organizational effectiveness of using DLs for legal research. This is conceptualized as the improvement of

63



organizational effectiveness via the DL’s level of search speed and its thoroughness over manual procedures of book searching. For this study, effectiveness of the California court system can be measured in many ways. One way is to count the number of cases completed by judges in each county in one year, rating them according to some predetermined threshold. Another is to measure the time saved and increased thoroughness gained by using DLs for research instead of books. A more informal means is based on assessment by informants, opinions or evaluations. We choose the latter approach to evaluate the influence of DL usage on organizational effectiveness.

Legal research

In preparing for a trial, attorneys for both sides may do legal research to interpret statutes and case law (the appellate and Supreme Court decisions) that might be beneficial for their arguments and evidence presentation in court. Since this law is constantly evolving, depending on the particular circumstances surrounding the case, lawyers need to keep abreast of recent changes to statutes and case law by reviewing the literature. Law libraries in courthouses generally contain state (pertaining to its jurisdiction) and federal legal reference books. These are used by district attorneys (DA)s (prosecutors for the government) and PDs in preparation for their cases. Civil and criminal defense lawyers purchase their own books or use a local law library.

A recent trend and the focus of this paper is the purchase of CD-ROMs and online DL services for legal research to replace or complement books. The CD-ROMs and online services provide corpuses which include federal and state law for all US. jurisdictions. From our research, we have found that some lawyers continue to use books, while others tend to combine digital and book research.

DL technology studied

The judges have access from their chambers to Lexis, an online legal research service from a personal computer (PC). The DAs are permitted access to Westlaw, an online legal research service, and 2 CD-ROM research PCs. PDs have access to CD-ROM research PCs with only appellate PDs authorized to use Lexis. A group of PDs (35) in City Y are part of an experiment where each has a PC on their desk networked into a CD-ROM tower containing LawDesk and Shepard’s Citations. The DA’s central library also has a PC linked to this tower.

The online research services provide the following legal information: California case (or statute) legal research, Shepard’s California and United States citations (up-to- date rulings on cases and statutes), federal case law

research, Witkin treatises (summaries of California law), out-of-state legal research, newspaper (or other news) research, and corporate (Dun and Bradstreet) research. The LawDesk CD-ROMs contain the California cases and (full text), the Shepard’s California citations, and the Witkin treatises (full text) and Deering’s codes (full text of California statutes).

The LawDesk CD-ROMs containing the California cases (California Reports and California Appellate Reports) represent the full text for approximately 800 volumes. The Shepard’s citations comprise about 10 volumes since these do not contain full text but indices into cases and statutes; Witkin’s summaries comprises 35 volumes and Deering’s codes represent about 200 books. Typically, lawyers will start research using Witkin’s summaries and then proceed to the California Reports and California Appellate Reports for more details. The CD- ROMs and online services also contain “slip opinions”, decisions that have been published in a daily newspaper but are not yet officially in print. Online services can have this information within 24 hours, while CD-ROMs are updated monthly. These unpublished decisions qualify as precedent law in court.

A LuwDesk search for “wrongful termination” yields 1190 cases in less than 10 seconds. Upon selecting one of those cases, the full text appears with hypertext links to other cases and laws on the CD-ROMs. Imagine looking through 800 books for all cases which mention “wrongful termination”, Thus, this technology provides a powerful research tool enabling searches which are not possible when just using books.

Shepard’s citations are references in a later authority to an earlier authority. The “shepardizing” procedure is important because it can show a legal professional whether or not a cited case has been overruled or reversed. The manual procedure for this is time-consuming and prone to omissions. With the use of CD-ROMs, citations are found in seconds.

DL organizational usability concept

Organizational usability varies with different organizations. It depends on the usability of the DL design itself and how well the DL characteristics complement the organizational characteristics.

The DL characteristics include:

l Interface Usability Dimensions (outlined in next section).

l Utility - Adequacy of system’s functionality to do what is needed.

l User’s Manuals - Quality of content and style of manuals.

64



l Phone Consulting- Level and quality of vendor/designer consulting sevices, usually by phone

l Training Services - Availability and quality of training services.

Organizational characteristics of an organization which influence organizational usability include:

l Access Points - The spatial distance between access points.

* Administrative Computing Knowledge - Computing administration responsible for selecting and implementing computer technology.

l End-user Computer Competence - Overall experience level of end-users with computer usage (can be a range). Very low means no former computer experience; very high means ability to write computer programs.

l Technical Support - Level of technical support offered for hardware problems.

l Training and Consulting - Availability and quality of in-house training and consulting for information systems.

Interface usability dimensions

Nielsen [8] identified five attributes of a system% interface usability. While there is some controversy in the HCI field about which specific set of interface characteristics to treat as most important, Neilsen’s list is sufficiently good for making our key contrasts between interface usability and organizational usability. He suggests that these attributes can be evaluated through usability testing relative to certain users and tasks. These dimensions include:

1. Learnability -- The system should be easy to learn such that a person can quickly begin using it.

2. Efficiency -- Ability of person to use the system efficiently after learning to use it.

3. Memorability -- Capability of a person to easily remember how to use the system after not using it for some period.

4. Software Reliability 1 -- System should have low error rates with few user errors, few catastrophic errors, and easy recovery from common errors.

5. Subjective Satisfaction -- Refers to how pleasant the system is to use.

According to Nielsen (1993), these interface usability dimensions are usually measured by having a number of test users (selected from a representative sample of the user

lThis dimension is called “Errors” by Neilsen (1993).

population) use the system while performing a predetermined set of tasks. Sometimes, actual users performing regular job duties are utilized for usability testing. With either technique, the usability is measured relative to specific users and specific tasks. By utilizing these precise attributes, the more abstract concept of usability can be measured. We use them here as categories to approximate the interface usability of the DLs studied.

Organizational usability dimensions

Issues of organizational design have been identified and faced by researchers and practitioners in neighboring fields, such as Information Systems and Computer Supported Work. Some researchers have addressed technical issues related to organizational design [14,15]. Others have focused on social aspects of design [lo, 161 and Bullen and Bennett [173 have looked at both technical and social impacts of organizational design on groupware usage. These studies suggest that people must be examined in their workplaces to understand the work practices which support DL usage. Research [18, 191 shows that organizations would get more from their original investment in computer systems, if they assessed the social infrastructure for supporting computer use as well as specific skills people need to use them.

We present four organizational usability dimensions from two perspectives - technical and sociotechnical. The technical perspective relates to issues which a DL designer or configurer can address to improve organizational usability. Whereas, the sociotechnical perspective is concerned with social dimensions of organizational usability. These social issues are harder to alter than technical ones and may require organizational change to improve organizational usability.

The technical dimensions of organizational usability include:

1. Accessibility. Ease with which people can locate specific DL systems, gain physical access and electronic access to their electronic corpuses.

2. Integrability into work practices -- How smoothly the system fits into a person’s or group’s work practices, including access to complementary computing resources (i.e., communication lines, scanners, printers) to facilitate work-flows -- how people actually prefer to work.

3. Social-organizational expertise -- The extent to which people can obtain training and consulting to learn to use DL systems in combination with existing computer systems and can find help with problems in usage.

65



4. Hardware Reliability -- DL systems should b-e reliable with easy recovery from common problems (with the possible assistance of support St&-).

Sociotechnical perspective

The same dimensions can be used to assess sociotechnical implications for organizational usability and pertaining to the following:

1. Accessibility - Social role influence on access privileges.

2. Integrability - Social incentives for purchasing DLs. Administrative control over expected use of DLs.

3. Reliability - Does the use of DLs for research make information more reliable to intended readers?

4. Social-Organizational Expertise - Influence of “informal” help from colleagues.

For the simplest of tasks, LawDesk and the online services show adequate learnability so that novice users can easily learn to do a basic search. For a casual user, memorability of LawDesk is high. However, the Shepard’s CD-ROM has low learnability and low memorability - its interface is clumsier and more error- prone than the others. For more complicated searches in LawDesk involving Boolean connectors, the learnability would be very low to low, since this involves special character entries that need to be studied in the user’s manual. Efficiency ratings are very high for LawDesk and Lexis and slightly lower for Shepard’s due to its more complicated interface. Software reliability is very high for the Windows version of LawDesk and Shepard’s. See [13] for an assessment of the DOS version of these systems in which numerous errors occur.

User’s manuals

User’s manuals for LawDesk contain tutorials which are difficult for novice computer users. In fact, most

Table 2. Characteristics of legal research DLs*

DL Characteristics

User Interface Dimensions Learnability Efficiency Memorabilitv

LawDesk Shepard’s Lexis (CD- (CD-ROM) (Online)

ROM)

3 2 3 5 3 5 4 2 4

Software Reliability 5 (Win) 5(Win) 5 2(DOS) 1 (DOS)

Subjective Satisfaction 3 2 3 User’s Manuals I 2 I 3 I 2 Training Services 2 1 ? Phone Consulting I 4 I 4 I 4 Utility 5 5 5

* 1 = very low, 5 = very high.

DL characteristics ratings

Table 2 shows ratings for DL characteristics for the CD-ROM and online services. These ratings were derived by one of the authors, M. Elliott, from her participant observation, interviews and hands-on experience (for CD- ROMs). Although these figures are subjective ratings, they serve to illustrate comparative analyses.

Interface usability dimensions

people interviewed had never looked at the user’s manual, but instead learned the system from in-house training - either by a colleague, librarian, or class.

Training services

Training services for LawDesk consists of a vendor representative giving a one-hour training session on how to start the program and how to do basic commands. In the criminal system studied, the PDs, DAs and judges have access to periodic training classes which successfully train people for all systems. Lexis offers an online tutorial

66



which one judge found to be very helpful. Neither judge Table 3 represents the ratings of LuwDesk, Shepard’s, and had ever had contact with a vendor training session so that the online services along technical dimensions of

Table 3. LawDesk/Shepard’s/Online - Technical dimensions of organizational usability. *

*1= very low, 5 = very high

field is not rated.

Phone consulting

Phone consulting services for all DLs are of high quality. However, these services are only available during normal working hours. Since many legal professionals work long hours or on weekends, afterhours phone consulting would be helpful.

Utility

The utility is very high for these systems in that they replace hundreds of books and enable searching vast amounts of information in a reasonable amount of time. These same searches performed manually would take much longer and in some cases, would be impossible to complete within a lifetime (e.g. searching all 800 of the books on LawDesk for a phrase). Also, the online services often post Supreme Court case law 24 hours after the decision. CD- ROM DLs are updated monthly to include these decisions prior to their being printed in books. So even though the interfaces may be clumsy and user’s manuals are lacking in preciseness, attorneys praise them for their expedience and comprehensive quality in information retrieval.

Organizational usability assessment of DL legal research: technical perspective

organizational usability. LawDesk and Shepard’s are rated together since they are often used in concert for research purposes and from the same location. Online access is also included for the DAs. Since only appellate PDs use online services, it is not included in this analysis. The judges use the online service Lexis solely (no CD-ROMs) at the time of this writing.*

These ratings are approximations derived from the authors’ participant observation, interviews and documentation.. We show the concept of organizational usability with this empirical data.

Accessibility

Research has shown that greater personal computer (PC) availability is associated with increased usage [20]. Our research correlates with Gogan’s (1991) study of PC use by auditors in two large public accounting firms. We found that the PDs, with LawDesk and Shepard’s available from their desk, are more apt to use these tools than DAs who have to go to the library . Judges have ready access to Lexis with a PC in their chambers.

Integrability

2 Judge B had just obtained access to the same networked CD-ROM available to the PDs and DAs, but he had not yet attempted to use it.

67



The administrative computing knowledge for the PDs and DAs is average. This rating is based on the fact that the DA’s office recently provided computers to head deputies, most of whom rarely use them. These computers could have been more useful to attorneys experienced with computer usage. In the PD’s CD-ROM experiment, adminisuators have been promising communication software to the PDs for a year. The PDs’ computers are hardwired for networking - something that would help them with their case management program - but the software has yet to be installed.

project requiring federal law. A judge’s social position of moderating and deciding legal disputes is probably a factor in decisions regarding allocation of resources.

End-user competence varies for both PDs and DAs. For example, one PD has created a case management database system written in File Maker Pro. The judge’s skills are average. Not surprisingly, those attorneys who have average to high computer competence are more apt to use DLs for research than those with low levels.

Similarly, the social role of DAs in the criminal courts as “fighters of criminals” versus PDs as “defenders of criminals” appears to influence funding for computer equipment in general and research DL access in particular. The PDs portray a picture of paucity of supplies and computer equipment. At the same time, the DA’s office is being supplied with computers for head deputies (managers of DAs) networked together with Lotus Notes software for communication. In addition, the DAs are supplied with a librarian and librarian’ s assistant to give continuous support for DL research, while the PDs have no librarian. The City X experiment with 35 PDs given a computer is an interesting anomaly; however, the funds for this project were part of a special research project.

Reliability Integrability

Technical support for the DAs is above average. No one complained of excessive “down” time during computer use. The judges have very high level support with a support staff member regularly monitoring their computer performance.

Social-Organizational expertise

Both PDs and DAs have opportunities to take classes at a training center located about 30 miles from the courthouse. However, the DA’s training for DL research usage is slightly higher due to its in-house librarian who regularly offers training assistance. Judges too have access to classes at the training center, but they have learned Lexis mostly from an online tutorial.

Organizational usability assessment of research DLs: sociotechnical perspective

Cultural models. Anthropologists have successfully used the concept of “cultural models” to understand the cultural constructs through which people view their worlds and how they internalize these constructs [21, 221. For example, anthropologists have used cultural models to study romance, marriage, parenthood and success of women and men from different class and ethnic backgrounds of people [21]. Shared knowledge, including values and assumptions, within a group is part of what anthropologists call “culture”. Our connection with “culture” includes how we make sense of different situations , and also, how we interpret actions as pertinent to any given situation [23]. Cultural models are “presupposed, taken-for-granted models of the world that are widely shared (although not necessarily to the exclusion of other, alternative models) by the members of a society and that play an enormous role in their understanding of that world and their behavior in it [22].”

Accessibility

Social position in the courts influences the allocation of computer technology. The elevated social role of judges enables them to have online access to Lexis at any time, even from home via a modem. DAs are discouraged from using the online systems due to the high costs. In fact, only PDs from the appellate division are authorized for online research. Most PDs, except those involved in the City X experiment, need to visit the library for CD-ROM use. However, only California statute and case law are available on CD-ROMs. This could give the judges an advantage over PDs and DAs when completing a research

Organizations are also viewed as cultures with a set of socially established structures of meaning which are accepted by its members [24]. Organizational cultural models are often used to examine current organizational work practices to understand how to change them. These models of behavior become cultural models when they are taken for granted within an organization or professional community as THE way that “things are done around here.” We use this perspective to explain why some of the attorneys expressed distaste for computers, and how those attitudes might be altered.

In this study, we found two camps of legal professionals with diverse attitudes toward computer usage. One group viewed computer technology with much enthusiasm with

68



comments like “I can’t live without my computer” (PD) and “If it [my computer] stopped working, I’d think, ‘Am I going insane?” (PD). For a detailed comparison of these two antipodal postitions, see [13]. Another viewed computer use with derision:

“I’m a Luddite. I’m just not machine-oriented.” (Criminal defense attorney)

“I don’t have interest. [in learning computers]. I’m not mechanical.” (PD)

[story from staff computer support about DAs:] ” Head deputies now have computers with Lotus Notes. My boss sent a note [in email] to all bureau directors . . . He attached a Word Perfect file so they wouldn’t have to retype. He wanted the directors to cc to all head deputies. Several secretaries have called me because they don’t know what to do with email. They said, They have this thing in their machine. I’m supposed to retrieve it but I don’t know how.‘... I tried to tell the bureau directors but they say, ‘I won’t use it.’ They don’t want to.”

These comments illustrate attitudes about the type of work which these attorneys prefer. They simply do not view machines as part of their world. Their cultural models of how legal professionals perform their work do not include using a computer. Statements likr “I’m a Luddite” and ” I’m not mechanical...” and the refusal of bureau chiefs to learn to use the computer illustrate the negative attitude that these attorneys have toward using technology. Some refuse to learn to use even the CD-ROM for research, claiming that the books are adequate for their purposes. However, the PD and criminal defense attorney both told me that if someone were to give them a computer and show them how to use it (one-on-one), they would probably be interested. This shows that perhaps these attitudes can be altered with the understanding of this resistence to change.

The organizational culture of attorneys and judges has been associated with books from ancient times. It is only in the past decade that electronic text has altered this strongly enculturated book-oriented society. A head deputy district attorney addressed this resistance to change:

“Well, you are dealing with a generation of people who are computer illiterate.... For researching a legal issue on a limited amount of time, the attorney will go to what is easy and what they are comfortable with. there must be a combination of books and CD-ROMs. You’ll never replace books.”

The law librarian for the DA’s office expressed problems with training attorneys to use computers:

“...often I suggest the CD-ROM and they resist learning something new. Like shepardizing, I can’t get people to use the CD-ROMS.”

The head deputy points out that lawyers will go to “what they are comfortable with” and the law librarian explains that she can’t get people to use the CD-ROMs. Perhaps one way to train people with these attitudes is individual training with constant reinforcement until they have mastered the basics. They may feel intimidated by a computer and, in a large class, might be afraid of looking like a failure - losing the image of the competent attorney who fights oral battles in court.

Coercive incentives for DL use. Judge A from Court A doesn’t like Lexis very much even though he has access to it from his chambers including a personal printer. He especially finds the use of Lexis impractical for the “shepardizing” procedure. Recently, the court administrators made the decision to eliminate the purchasing of the Shepard’s Citations books for each judge, requiring them to use Lexis for all current “shepardizing”. Judge A likes to read something from a Shepard’s Citations book from his bench, and he generally knows exactly which code he is referencing. It takes more time and is inconvenient for him to excuse himself in the middle of a trial, go to his chambers, dial into Lexis, print out a citation, and take it back to the bench to be read. By nature of this coercive incentive, the judge has no choice but to use the computer. He is even more concerned with future coercion techniques planned by the administration:

“I use Lexis but I don’t really like it. The future plan for the LA Superior Courts is that there will be one library of books per floor and each judge will have CD-ROMs in their chambers with volumes of books on them. This will definitely cause problems!”

Even though the judge does not view the DL research as a pleasurable experience, he will be forced to eventually use the CD-ROMs. These administrative decisions are being made due to financial concerns. CD-ROMs are much less expensive than law libraries and judges will need to adjust to this sociotechnical change in their style of work. Judge A might find this experience more amenable if he were given a CD-ROM at the bench.

Reliability

69



Legal research with LawDesk and Shepard’s DLs can be considered more reliable than manual methods due to its thoroughness and up-to-date searches. Online research and CD-ROM DLs enable quick access to recent decisions before they are published. Both contain “slip opinions”, decisions that have been published in a daily newspaper but are not yet officially published. This unpublished material, which can be used to support arguments in court, gives credence to the promotion of DL information as more reliable. Judge B praised the research tools as extremely valuable:

“Lexis is a powerful legal tool... I can do sophisticated research with Lexis.” (Judge B)

Attorneys feel more prepared for a trial when they have exhausted all searches for relevant research. The DLs promote the increased reliability of the research material.

Social-Organizational expertise

Informal “help” seems to work best for computer training legal professionals. The PDs with computers on all their desks can ask a colleague for assistance throughout the day. Most attorneys are either lost when attending formal classes or do not want to allocate time to take the class. Research has shown that more is needed to have widespread usage of computer systems than merely implementing it [18, 191.

Discussion and conclusion

The research makes two contributions:

First, we introduced the concept of “organizational usability” from both a technical and sociotechnical perspective. We presented four dimensions of organizational usability and used them to assess the organizational usability of DLs in a California Superior court. The research has shown that sociotechnical dimensions shape the organizational usability of DLs as much as technical dimensions. We hope this study provokes thought and action among configurers and designers to consider organizational usability in future DL designs.

Second, a fantastic DL interface doesn’t necessarily lead to high usage levels. In fact, without the incentive for use, the DL’s design quality and organizational usability might be very high, but not be useful to its members. Orlikowski (1992) has shown that the successful introduction of groupware into an organization depends on people’s mental models understanding its collaborative nature. In addition, incentives or norms for cooperating or sharing expertise

need to be in place. Groupware innovation alone can not cause this cooperation to happen.

In a similar fashion, this study has shown that without mental models of the utility of DL research tool and incentives for using it, attorneys do not widely embrace this technology. For example, the technical organizational usability scores for DL usage by judges were higher than scores for DAs and PDs in the dimensions of accessability, integrability, and equipment reliability. However, Judge A preferred using books over Lexis. As discussed in the section on cultural models, some people have a mental picture of an attorney or judge’s role which does not include using “machines”. Whether a DL has a good or bad interface, these people’s attitudes will probably preclude its US?.

The California courts are in the middle of a transition from low technology usage to high. In the state of California, there is an initiative in place, administered by a Court Technology Committee, to strengthen the use of technology in the courts [26]. The primary mission of this committee is promoting, coordinating and facilitating the application of technology to the work of the California courts. In order to plan the most effective implementation of court technology over the 208 courts and 1, 553 authorized judgeships in the state of California, empirical studies are needed to explore optimum usability of deployed systems. This is an ideal time to study the organizational usability of DLs in the courts.

References

[II

PI

[31

[41

(51

[61

[71

Kittle, P. (1988). MEDLINE on CD-ROM: A review of six products, Laserdisk professional, l(3), 18-28. Bantliff, B. A. (1992). Introducing CD-ROMs into a law library: Administrative issues and concerns. Law Library Journal, 84m 725740. Komowski, J. (1995). CD-ROM for the law office: Moving beyond the potential of books and paper. Los Angeles Lawyer, 17(1 l), 62-64. Shields, Mark (Ed.). (1995). Work and Technology in Higher Education: The Social Construction of Academic Computing. Hillsdale, NJ: Lawrence Erlbaum and Associates. Aaron, A.. The struggle to get California’s courts on-line. California Law Business, Monday, January 9, 1995, 18c. Gould, J. D. and Lewis, C. (1991). Making usable, useful, productivity-enhancing computer applications. Communications of the ACM 34, 74- 86. Kling, R. and Elliott, M. (1994). Digital library design for usability. In Schnase, J., Leggett J.,

70


Proceedings of the 29th Annual Hawaii Intern .ational Conference on System Sciences - 1996

@I

191

lm

[Ill

WI

[I31

[I41

v51

P61

iI71

1181

r.191

WI

WI

Furuta, R., and Metcalfe, T. (Eds.), Proceedings of Digital Libraries ‘94 Conference (pp. 146-155). College Station, Texas. Nielsen, J. Usability Engineering. (1993). Boston, Mass.: Academic Press, Inc. Eason, K. D. (1988). Information technology and organisational change. London: Taylor & Francis. Markus, M. L. and Robey, D. (1983). The organizational validity of management information systems. Human Relations, 36(3), 203-226. Fox, E. A. (1993). Source Book on Digital Libraries. Version 1.0. Prepared for and sponsored by the National Science Foundation, Washington, DC., Blacksburg, Va.: Virginia Polytechnic Institute & State University.

Yin, R. K. (1994). Case study research: Design and methods. Thousand Oaks, CA: Sage Publications. Elliott, M. and Kling, R. (1995). Organizational usability of digital libraries: Case study of legal research. Working Paper. Irvine, CA: Center for Research on Information Technology and Organizations, University of California. Culnan, M. (1983). Chauffeured versus end user access to commercial databases: The effects of task and individual differences. MIS Quarterly, 7,55-67. Trauth, E. M. and Cole, E. (1992). The organizational interface: a method for supporting end users of packaged software. MIS quarterly 16(l), 35-53. Forsythe, D. (1992). Blaming the user in medical informatics: the cultural nature of scientific practice. Knowledge and Society: The Anthropology of Science and Technology ,9,95-l 11. Bullen, C. V. and Bennett, J. L. (1991). Groupware in practice: An interpretation of work experiences. In C. Dunlop and R Kling (Eds.), Computerization and controversy (pp. 257-287). Boston, MA: Academic Press, Inc. Kling, R. (1992). Behind the terminal: The critical role of computing infrastructure in effective information systems’ development and use. In Cotterman, W. and Senn, J. (Eds.), Challenges and strategies for research in systems development,. London: John Wiley. Markus, M. L. and Keil, M. (1993). If we build it, they will come: designing information systems that users want to use. Technical Report. Claremont, Ca.: The Claremont Graduate School. Gogan, J. L. (1991). Should “personal” computers be personally allocated? Journal of management information systems, 7(4), 91-106. D’Andrade, R. G. and Strauss, C., Eds. (1992). Human motives and cultural models. Cambridge, Mass.: Cambridge University Press.

[22] Quinn, N. and Holland, D. (1987). Introduction. In Holland, D. and Quinn, N. ( Eds.), Cultural models in language and thought (pp. 3-40). New York: Cambridge University Press.

[23] Geertz, C. (1992). Thick description: Toward an interpretive theory of culture, in The Interpretation of Cultures. New York: Basic Books, Pp. 3-30.

[24] Schein, E. H. (1992). Organizational culture and leadership. San Francisco, CA: Jossey-Bass Publishers.

[25] Orlikowski, W. J. (1992). Learning from notes: Organizational issues in groupware implementaion. In CSCW 92 Proceedings.

[26] Judicial Council of California. (1995). Report of the court technology task force. administrative office of the courts, Sacramento, CA, January 25, 1995.

Appendix A. Summary of questions (These are general catergories for attorneys and judges. Each interviewee had unique qualities ).

General work

Type of work and amount delegated. Percent of time spent on research. Effects of California Three Strikes Law on work load.

General Computer Usage

First experience with a computer. Training needed/desired for first experience and now. Home computer. Work-related tasks on computer. Attitude if computer at work is removed.

DL usage

How learned - training or self-taught? Attitudes/beliefs concerning organizational dimensions -

technical and sociotechnical. Frequency of use. Modification to work. Use of paralegals. Request for demonstration of DL.

Future of computers in courts

What predictions can you make about future of computing in the courts?

Attidtudes towards new technology - realtime court reporting, DLs, multimedia evidence presentation.

71


organizational usability of digital libraries in the...

Documents