usability testing for learnability and efficiency...

USABILITY TESTING FOR LEARNABILITY AND EFFICIENCY ON UUM DIGITAL LIBRARY SYSTEM

ASNA KHAIRUN BINTI ABDUL RAHMAN

UNIVERSITI UTARA MALAYSIA

2012

USABILITY TESTING FOR LEARNABILITY AND EFFICIENCY ON UUM DIGITAL LIBRARY SYSTEM

A project submitted to the Dean of Research and Postgraduate Studies Office in partial

Fulfillment of the requirement for the degree of

Master of Science (Information and Communication Technology)

Universiti Utara Malaysia

By

Asna Khairun Binti Abdul Rahman

(807624)

Copyright © Asna Khairun Binti Abdul Rahman, 2012. All rights reserved.

In presenting this p

Science in Information

Utara Malaysia, I agre

inspection. I further a

whole or in part, for s

absence by the Dean o

understood that any c

financial gain shall not be

that due recognition sh

scholarly use which m

Requests for p

in whole or in part, shou

PERMISSION TO USE

this project in partial fulfillment of the requir

tion and Communication Technology (MSc. IC

ree that the University Library may make it free

agree that permission for copying of this thesis in

scholarly purpose may be granted by my super

of Awang Had Salleh Graduate School of Ar

copying or publication or use of this thesis or

ll not be allowed without my written permission.

nition shall be given to me and to University Utara M

h may be made of any material from my project p

permission to copy or to make other use of m

t, should be addressed to

Dean of Awang Had Salleh Graduate School

College Arts and Sciences Universiti Utara

Malaysia

06010 UUM Sintok

Kedah Darul Aman

Malaysia

i

rements for a Master of

CT) from University

freely available for

sis in any manner, in

ervisor(s) or, in their

rts and Sciences. It is

parts thereof for

ion. It is also understood

Malaysia for any

t paper.

materials in this thesis,

hool

a

Kepentingan p

pembelajaran di dalam bilik d

menyediakan sistem p

pantas dan cekap kep

memberi perkhidmata

Walaupun penggunaa

sejumlah pengguna-p

perpustakaan digital. Uj

satu daripada ciri-ciri ut

untuk mengkaji kebol

Malaysia (UUM). Ke

kebolehgunaan yang t

kualitatif dan kuantita

prestasi dan borang-bo

untuk mendapatkan kol

data kuantitatif dan ku

dipelajari dan digunak

ABSTRAK

n penggunaan laman web sebagai proses dalam p

m bilik darjah meningkat secara mendadak. L

m perkhidmatan dan sumber untuk mendapatkan m

pada pengguna. Perpustakaan digital adalah la

an kepada pengguna di seluruh dunia dan diter

aan sistem perpustakaan digital semakin menin

pengguna yang membuat rungutan mengenai k

l. Ujian kebolehgunaan bagi sistem perpustakaa

i utama untuk mencapai kejayaan. Projek ini m

lehgunaan bagi sistem perpustakaan digital di Univ

eupayaan untuk belajar dan kecekapan adalah

telah dipilih bagi projek ini. Ujian kebolehgun

atif telah digunakan dalam projek ini. Tugas-t

borang soal selidik telah diagihkan kepada respon

n koleksi data kuantitatif dan kualitatif. Hasil ka

n kualitatif memberi rumusan bahawa sistem ini s

kan.

ii

m persekitaran

Laman web

n maklumat secara

aman web yang

erima secara meluas.

ningkat, masih terdapat

i kebolehgunaan

aan digital adalah salah

ini memberi tumpuan

l di Universiti Utara

h ciri-ciri

unaan bagi koleksi data

tugas untuk mengukur

sponden-responden

ajian daripada koleksi

m ini senang untuk

The interest in using

environment increase

users to get information

users and become wid

used, there are a numb

usability test of digita

project focuses to look i

Malaysia (UUM). Lear

chosen for this projec

had been applied for this p

been distributed to respond

results from quantitative

use and to learn.

ABSTRACT

st in using the Website in the classroom as part of

se dramatically. Website provides an efficient a

tion and service. Digital library is a website tha

widely accepted. Although the digital library syst

number of users complaining about usability of di

al library system is one of the key features for

s to look into usability of digital library system in Univ

arnability and efficiency are usability attribut

ect. Usability testing on qualitative and quantita

this project. Tasks on performance measure a

spondents for quantitative and qualitative data

tive and qualitative data collection showed tha

iii

t of the learning

and fast access for

at serving worldwide

stem increasingly

igital library. The

or its success. This

m in University Utara

ibutes that has been

ative data collection

and questionnaires had

ta collection. The

at the system is easy to

Firstly, praise

perseverance and stre

parents, siblings and fr

dedicate my endless g

all his efforts, guiding

Azizah Ahmad, my p

comments throughout th

directly or indirectly in

Allah SWT continue

Asna Khairun Binti Ab

January 2012

ACKNOWLEDGEMENT

and thanks to Allah SWT because of His ble

ength to accomplish this project. Secondly, I would l

friends for their support in all aspects. Thirdly

gratitude to my project supervisor, Dr. Mohd

g, advising and teaching me all this while. The

project evaluator, thank you for all the helpful

hout this master project. Lastly, thank you for those

in this project. I am really appreciated all you

to bless all of you. Amin.

inti Abdul Rahman

iv

essing with

would like to thank my

y, I would like to

. Mohd Syazwan Abdullah for

. The same goes to Dr.

ul and supportive

those who are involved

ur supports and May

PERMISSION TO U

ABSTRAK

ABSTRACT

ACKNOWLEDGEME

TABLE OF CONTENT

LIST OF TABLES

LIST OF FIGURES

LIST OF ABBREVIA

CHAPTER 1: INTRO

1.1Background

1.2 Problem S

1.3 Research Qu

1.4 Objectives

1.5 Scopes and

1.5.1 S

1.5.2 L

1.6 Significan

1.7 Summary

TABLE OF CONTENTS

O USE

EMENT

ENTS

ATIONS

ODUCTION

ound

Statement

Questions

s

nd Limitations

Scopes

Limitations

nce of Study

v

Page

i

ii

iii

iv

v

ix

xi

xii

1

2

3

3

4

4

5

5

6

CHAPTER 2: LITER

2.1 Introduction

2.2 Digital Lib

2.2.1

2.2.2

2.3 Integra

2.4 Usabili

2.4.1

2.4.2

2.4.3

2.4.4

2.5 Summ

CHAPTER 3: METHO

3.1 Fact G

3.2 Questionn

3.2.1

3.2.2

3.3 Prelimin

3.4 Condu

3.5 Sample

RATURE REVIEW

tion

ibrary

Advantages of Digital Library

Evaluation of Digital Library System

rated Library System (ILS)

bili ty

Learnability

Efficiency

Usability Evaluation

Usability Testing

ummary

3: METHODOLOGY

Gathering

stionnaire and Task Construction

Questionnaire

Task Construction

liminary Study

onducting Pilot Study

mple Selection

vi

7

7

9

10

10

11

13

13

14

21

22

25

25

25

27

27

28

28

3.6 Data c

3.7 Data An

3.8 Recomm

3.9 Summ

CHAPTER 4: RESUL

4.1 Quantit

4.1.1

4.1.2

4.1.3

4.2 Qualita

4.2.1

4.3 Summ

collection

Analysis and Evaluation

ommendation

ummary

LTS AND FINDINGS

ntitative Data

Analysis of Pre-Test Questionnaire

4.1.1.1 Gender

4.1.1.2 Age Group

4.1.1.3 Three Main Features of Online Resou

4.1.1.4 Rating for Overall Learning Operation

4.1.1.5 Rating for Overall Efficiency Aspec

Analysis of Task

Analysis of Post-Test Questionnaire

4.1.3.1 Learnability

4.1.3.2 Efficiency

ative Data

Analysis of Post-Test Questionnaire

ummary

vii

29

29

30

30

31

31

31

32

sources 33

tion 33

t 34

35

37

37

38

40

40

46

CHAPTER 5: CONCLUSIONS AND RECOMMENDATIONS

5.1 Discussion of Findings 47

5.2 Problems and Limitations 48

5.3 Recommendations 49

5.4 Contributions to the Study 50

5.5 Conclusion 50

REFERENCES 51

APPENDIX A: PRE-TEST QUESTIONNAIRE 56

APPENDIX B: POST-TEST QUESTIONNAIRE 1 58

APPENDIX C: POST-TEST QUESTIONNAIRE 2 61

APPENDIX D: TASK 63

viii

Table 2.1: Advantage

Table 2.2: Detailed O

Table 2.3: Basic Elem

Table 3.1: Methodolo

Table 3.2: Four-Point

Table 3.3: Positive and

Table 4.1: Frequency

Gender


Age


Features


Learnability


Efficiency

Table 4.6: Time to Com

Table 4.7: The Analy

Table 4.8: The Analy

Table 4.9: The Conte

Table 4.10: The Cont

LIST OF TABLES

es of Digital Library

d Overview of Usability Evaluation Method

ments of Usability Testing

thodology

oint Likert-Scale

nd Negative Questions

y Distribution and Percentage of the Responde





omplete Each Task

ysis on Learning to Operate UUM Digital Libr

ysis on Efficiency of UUM Digital Library Sy

ent Analysis for Question 3.1

ontent Analysis for Question 3.2

ix

9

17

22

23

26

26

ents According to

32

ents According to

32

ents According to

33

ents According to

34

ents According to

34

35

rary System 37

ystem 39

41

42







x

43

44

45

Figure 1.1: The Scree

Figure 2.1: The Usabili

Figure 3.1: Tasks

Figure 4.1: The Scree

LIST OF FIGURES

creen of UUM Digital Library System

bili ty Evaluation Process on Website

creen of Error Page on UUM Digital Library Syst

xi

4

15

27

stem 36

CAS

FAQ

HCI

HTML

ILS

ISO

LMS

PDF

PUEU

QUIS

UUM

LIST OF ABBREVIATIONS

College Arts and Sciences

Frequently Asked Questions

Human Computer Interaction

Hyper Text Markup Language

Integrated Library System

International Standards Organization

Library Management System

Portable Document Format

Questionnaire for Perceive Usefulness of Ea

Questionnaire for User Interaction Satisfaction

University Utara Malaysia

xii

ase-of-Use

tion

Chapter one p

also includes brief ex

objectives, scopes and limit

1.1 Background

The digital lib

user. Most libraries de

library`s doors closed

provides hours of opera

online thesis, and oth

system service just in the

increase the percentage

There are man

system. It creates bett

users can use the same

will create advantages su

users can use the same

(2009), usability and t

efficiency of the user’

their task goals when acquiring resources in relation to successf

task within a reasonable time (Buchanan, 2009).

CHAPTER 1

INTRODUCTION

provides an overview of digital library and the

xplanation of background, problem statement,

nd limitations, and significance of the study.

l library systems are as common as books and ser

epend on their website to give services for us

d as well as they are open. The digital library s

eration but it also allows user to search for online

nd others. It is user friendly because they can use d

just in the fingertip at anytime and anywhere. It would h

ge of users.

ny important benefits for users if they use this d

tter and faster resources and information servi

me resource, free to users, and others. The digit

s such as faster resources and information ser

me resource, free to users, and others. Accordin

nd the usefulness of a digital library depends on

er’s experience. It is important that a user is able

their task goals when acquiring resources in relation to successful

task within a reasonable time (Buchanan, 2009).

1

e usability issues. It

research questions,

erving a worldwide

users when the

system not only

online catalogs,

digital library

t would help to

this digital library

vices, unlimited

ital library system

ervices, unlimited

ng to Buchanan

nds on effectiveness and

It is important that a user is able to accomplish

ul completing the

The contents of

the thesis is for

internal user

only

References

Arms & William Y. (

United States: The

Barnum, C. (2002). Usa

New York: Lo

Bartek, V. & Cheatha

Part 2: Examine

Retrieved Dece

http://www.m

2-examine-be

Bates, M. J., Wilde, D.

Humanities Sc

Project Repor

pp. 5-40.

Battleson et al. (2001

A case study, T

198.

Ben D. Harper & Kent

The Questionnaire

October, 2011,

Blandford, A. E., Stelm

digital librari e

(2000). Digital Libraries.

s: The MIT Press.

Usability Testing and Research.

ongman.

am, D (2003). “Experience Remote Usability

amine the Benefits and downside of Remote Usability

ecember 2011, from

http://www.mendeley.com/research/experience-remote-usa

enefits-downside-remote-usability-testing/

, D. N. & Siegfried (1995). S. Research Practice

cholars In an Online Environment. The Getty

rt No. 3. Library and Information Science Res

1). Usability testing of an academic library We

The Journal of Academic Librarianship, Vol. 27

nt L. Norman (1993). Improving User Satisfa

stionnaire for User Interaction Satisfaction Version 5.5

, 2011, from http://lap.umd.edu/quis/publications/h

lmaszewska & H. & Bryan-Kinns (2001). N. Use

es: a case study. In Proc. JCD. ACM Press (2001

51

Testing,

Usability Testing”.

ability-testing-part-

ces of

y Online Searching

search 17 (1995),

Web site:

Vol. 27(3), pp. 188-

r Satisfaction:

rsion 5.5. Retrieved

tions/harper1993.pdf

. Use of multiple

2001). pp. 179–188.

Borgman (2000). C. L

Access to information in

Press.

Branaghan, R. (1999)

Testing, Retr

fromhttp://www.b

Buchanan, S., Salako, A.

library. Library

Carol Smith (2006). Mul

Cognitive The

227–238.

Chin, J. P., Diehl, V.A., &

instrument mea

Proceedings of the

systems, pp. 213

Conyer, M. (1995). Us

Australian Journal of E

September, 2011,

Corgan, T., & Waiter

Cambridge: C

Davis. F. D. (1989). P

Acceptance of

Retrieved Sept

http://www.cb

L. From Gutenberg to the global information

ss to information in the networked World. Cambridge, MA: T

). Testing, One—Two—Three: Fundamentals of Usab

rieved December, 2011,

omhttp://www.branaghan.com/fun_utesting.htm.

ko, A. (2009). Evaluating the usability and usefuln

ibrary Review, 58(9), pp. 638-651

. Multiple Cultures, Multiple Intelligences: Appl

eory to Usability of Digital Libraries. Libri, 2006,

hl, V.A., & Norman, K. L. (1988). Development of

easuring user satisfaction of the human-comput

dings of the SIGCHI conference on Human factors in

, pp. 213-218.

. User and usability testing-how it should be und

ournal of Educational Technology, 11(2), pp. 38

, 2011, from http://www.ascilite.org.au/ajet/ajet.h

ers, N. J. (1994). Usability testing: what are we

Cambridge University Press.

Perceived Usefulness, Perceived Ease of Use,

f Information Technology. MIS Quaterly, 13(

ptember, 2011, from

ba.hawaii.edu/chismar/ITM704/Davis-TAM1989.pdf

52

infrastructure:

, MA: The MIT

ntals of Usability

fulness of a digital

Applying

ibri, 2006, vol. 56, pp.

nt of an

omputer interface.

tors in computing

undertaken?

, pp. 38-51. Retrieved

t.html

e learning?

, and User

(3), pp. 319-340.

TAM1989.pdf

Deddens Marcia (200

Retrieved Dece

http://net.educa

Dumas Joseph S.& R

UK, US: Intell

Ebling, M. R & John,

usability testin

systems: proce

2011, from http://p

Faulkner, C (1998). T

United States of

Garrett, J. R. & Lyons,

management s

44(8), pp. 468

Gore, P. & Hirsh, S.G.

site”. Online, Vol. 27, p

Graneheim U. H. & L

research: conce

Department of

112. Retrieved D

http://www.em

Hootman & Robert C

evaluation. Ba

ISO 9241-11, (1998),

terminals (VD

from http://www.it.uu.s

2002). Overview of integrated library systems.

ecember 2011, from

cause.edu/ir/library/pdf/DEC0201.pdf

Redish Janice (1999). A practical Guide to Usability

llect Ltd.

ohn, B.E. (2000). On the contributions of different

ng. Proceedings of the conference on designing int

cesses, practices, methods, and techniques. Re

http://portal.acm.org/

The Essence of Human-Computer Interaction.

s of America: Prentice Hall.

ons, P.A. (1993). Toward an electronic copyright

system. Journal of American Society for Information S

, pp. 468-73.

.G. (2003), “Planning your way to a more usable

, Vol. 27, pp. 20-7.

Lundman B. (2003). Qualitative content analy

cepts, procedures and measures to achieve trustwo

nt of Nursing, Umea University, Umea 90187, Sw

d December, 2011, from

meraldinsight.com.eserv.uum.edu.my/

C. (1992). Manual on descriptive analysis test

altimore: Library of Congress Cataloging-in-P

, Ergonomics requirements for office work with

VDTs): Part 11-Guidance on usability. Retrieve

om http://www.it.uu.se/edu/course/homepage/acsd/vt09/I

53

to Usability Testing.

nt empirical data in

signing interactive

etrieved October,

tion.

ht

nformation Science,

usable web

ysis in nursing

ustworthiness.

weden 24, 105–

ting for sensory

Publication-Data.

ith visual display

ed October, 2011,

ISO9241part11.pdf

Krippendorff Klaus (2004

United States:

Larry L. Constantine

Addison Wesl

Lazar, Feng & Hochh

Computer Inte

Lesk Michael (1997).

United States of

Liu Fang (2008). Usability

Wuhan Univer

Lucy A. Tedd & And

in a Global En

Maurer, D. (2004). W

Retrieved Sept

usability

Muller Tristan (2011)

library system.

perspectives. Vol. 27, pp.

http://www.em

Nielsen Jakob (1993)

United States: A

Norlin Elaina & Wint

Chicago and L

2004). Content Analysis: An introduction to i

s: Sage Publications Inc.

& Lucy A. D. Lockwood (1999). Software for Us

sley: Reading MA.

hheiser (2009). Research Method in Human

eraction. United Kingdom: John Wiley & Sons

. Practical Digital Libraries: Books, Bytes &

s of America: Morgan Kaufmann Publishers,

Usability Evaluations on Websites. School of Art

ersity of Technology, Wuhan, Hubei Province

drew Large (2005). Digital Libraries: Principl

in a Global Environment. Germany: K. G. Saur Verlag GmbH.

What is usability?

ptember, 2011, from www.steptwo.com.au/pa

). How to choose a free and open source integ

m. OCLC Systems & Services: International digital

Vol. 27, pp. 57-78. Retrieved December, 2011,

meraldinsight.com.eserv.uum.edu.my/

). Usability Engineering.

s: Academic Press.

inters CM (2002). Usability Testing for Library

London: American Library Association.

54

its methodology.

for Use.

ons Ltd.

Bucks.

Inc.

t &Design,

ce, pp.141.

iples and Practice

GmbH.

apers/kmc_whatis

grated

ational digital library

, 2011, from

ry Web Sites.

Oppenheim, Charles &

Journal of Information

Preece, J., Rogers, Y.,

human-comput

translated by Xi

Prescott, J., & Crichton

method. Procee

services: Mile

http://delivery

Rubin Jeffrey & Chisn

Plan, Design, and

Senapathi, M. (2005).

in Educationa

Vol. 4, pp. 61

Shackel & Simon J. R

Britian: Camb

WWWebster Dictiona

from http://www.m

& Smithson, David. (1999). What is the hybr

nformation Science. Vol. 25, pp. 97-112.

s, Y., & Sharp, H. (2002). Interaction Design: Bey

omputer interaction. Publishing House of Electroni

Xiaohui Liu, Jing Zhang, pp. 224-301.

hton.M. (1999). Usability testing: a quick, chea

ceedings of the 27th annual ACM SIGUCCS conf

ile high expectations. Retrieved September, 2011,

y.acm.org/

hisnell Dana (2008). Handbook of Usability Testing:

sign, and Conduct Effective Tests. Canada: John W

. A Framework for the Evaluation of CASE Tool

al Environments. Journal of Information Technology

Vol. 4, pp. 61-84.

Richardson (1984). Human Factors for Informati

mbridge University Press.

ary, (1999). Meriam –Webster. Retrieved Octob

om http://www.m-w.com.

55

rid library?

eyond

tronics Industry,

eap and effective

onference on User

, 2011, from

sting: How to

Wiley & Sons, Inc.

E Tool Learnability

hnology Education.

nformatics Usability.

tober, 2011,

usability testing for learnability and efficiency...

Documents