usability testing report (com5338)
TRANSCRIPT
USABILITY TESTING REPORT: STATE UNIVERSITY SYSTEM of FLORIDA BOARD of GOVERNORS TEAM DARTMOUTH: Lenore Messler & Whitney Payne COM5338-01 Fall 2012 Dr. Jonathan Adams
Team Dartmouth COM5338-01
Usability Testing Report
2
CONTENTS PART I: Project Overview
Executive Summary ---------------------------------------------------------------------------------------------- 3 Client Description ------------------------------------------------------------------------------------------------- 3
Background ---------------------------------------------------------------------------------------------- 3 Mission ---------------------------------------------------------------------------------------------------- 3 Values ------------------------------------------------------------------------------------------------------ 3
Audience Analysis ------------------------------------------------------------------------------------------------ 4 Primary Audience Segments ----------------------------------------------------------------------- 4 Secondary Audience Segments ------------------------------------------------------------------ 4 Conclusion ----------------------------------------------------------------------------------------------- 5
Technology Analysis --------------------------------------------------------------------------------------------- 5 Background ---------------------------------------------------------------------------------------------- 5 ADA Compliance -------------------------------------------------------------------------------------- 5 Minimum Requirements ----------------------------------------------------------------------------- 6 Operating Environment ----------------------------------------------------------------------------- 7
Website Description --------------------------------------------------------------------------------------------- 7 Background ---------------------------------------------------------------------------------------------- 7 Project ----------------------------------------------------------------------------------------------------- 7
PART II: Supplemental Design Documents
Design Overview/Approach ----------------------------------------------------------------------------------- 8 Global Conventions ---------------------------------------------------------------------------------------------- 8 Current Flowcharts ------------------------------------------------------------------------------------------- 8-12 Redesign Flowcharts --------------------------------------------------------------------------------------- 13-16 Storyboards --------------------------------------------------------------------------------------------------- 17-20 Navigation Table ------------------------------------------------------------------------------------------------ 21 Typeface/Font Specifications ------------------------------------------------------------------------------- 21 Color Table ---------------------------------------------------------------------------------------------------- 21-22 Image Table ------------------------------------------------------------------------------------------------------- 22 Button States ----------------------------------------------------------------------------------------------------- 22 Design Comps ----------------------------------------------------------------------------------------------- 23-24
PART III: Usability Tests & Results
Overview ----------------------------------------------------------------------------------------------------------- 25 Method -------------------------------------------------------------------------------------------------- 25 Equipment --------------------------------------------------------------------------------------------- 25 Test Participant Profiles --------------------------------------------------------------------------- 26
Findings ------------------------------------------------------------------------------------------------------------ 27 Test Task Results ------------------------------------------------------------------------------------ 27 Design --------------------------------------------------------------------------------------------------- 27 Navigation --------------------------------------------------------------------------------------------- 28 Other ---------------------------------------------------------------------------------------------------- 29
Usability Test Script ---------------------------------------------------------------------------------------- 29-31 Usability Test Consent Form -------------------------------------------------------------------------------- 32 Post-Test Questionnaire -------------------------------------------------------------------------------------- 33
REFERENCES --------------------------------------------------------------------------------- 34
Team Dartmouth COM5338-01
Usability Testing Report
3
PART I: PROJECT OVERVIEW
Executive Summary/Overview
The Board of Governors website serves to provide information about the State University System of Florida and the Board of Governors to a broad and varied audience. The mission of the Board of Governors is “To mobilize resources and diverse constituencies to govern and advance the State University System of Florida” (Board of Governors, 2008). The current website lacks design that aids usability and therefore fails to complete the mission of advancing the State University System of Florida. The Board of Governors wishes to make use of the website easier and more engaging. In accordance with the company’s values of continuous improvement and transparency, the new website will be regularly updated and will effortlessly guide users to the information that they seek.
Client Description
Background
The Board of Governors is the governing body for the State University System of Florida. The Board is comprised of seventeen members: fourteen appointed by the governor, the Florida Commissioner of Education, the Chair of the Advisory Council of Faculty Senates, and the President of the Florida Student Association. The Board of Governors was established in 2003 by a constitutional amendment. The State University System is comprised of 12 public universities and has over 300,000 students. The office of the Board of Governors has between 50-60 employees and is overseen by the Chancellor.
Mission “To mobilize resources and diverse constituencies to govern and advance the State University System of Florida.” (Board of Governors, 2008)
Values “To support and advocate for high-quality teaching, research and public service, we are committed to:
• Creativity, discovery and innovation • Student access, learning and success in the global community and
marketplace • Collaboration, respect and appreciation of diversity • Transparency, shared responsibility and continuous improvement”
(Board of Governors, 2008)
Team Dartmouth COM5338-01
Usability Testing Report
4
Audience Analysis
The target audience for the Board of Governors website can be separated into four primary segments, all of which are interested in higher education: 1) Prospective/Current students of higher education, 2) Board of Governors/Boards of Trustees members, 3) Legislators involved in higher education, and 4) Media reporting on higher education. Each audience segment has different needs and is comprised of different demographic that will be described in this document.
Prospective/Current Students of higher education Comprised primarily of 16-24 year olds located in Florida. Other members of this audience may be slightly younger or older, and/or be located outside of Florida. This audience is visiting the website to learn about two things: 1) information about universities in the System before they apply, or 2) information about higher education in Florida such as financial aid and the academic common market.
Board of Governors/Boards of Trustees members and staff
Comprised of members or staff of the Board of Governors and members of individual university’s Boards of Trustees. Age of this audience varies from the student representative to members who are approximately 70-80 years old. This audience is based in Florida and visiting the website primarily to search for Board of Governors policies, publications, and meeting dates/agendas.
Legislators involved in higher education Comprised of all members of Florida government with an interest in higher education. This audience will primarily visit the “Press Room” page, “Facts/Figures” page, and search for Board of Governors policies, publications, and meeting dates/agendas.
Media reporting on higher education
Comprised of all members of Florida and national media. Members of this audience will primarily visit the “Press Room” page, Board meeting information pages, and search for Board of Governors policies, publications, and meeting dates/agendas.
Secondary Audience Segments Parents of students in higher education
This audience is comprised of parents of prospective or current students and will visit similar pages as the Prospective/Current Students audience.
Faculty of higher education institutions This audience is comprised of faculty of higher education institutions, particularly those within the System.
Team Dartmouth COM5338-01
Usability Testing Report
5
Conclusion The content of the website and the varied audience segments constrain the voice of the website. As a government organization, it is important for the website to maintain a professional look and sound. Language used in the website will be very straightforward and serve only to provide clear information. In addition, because all target audience segments are busy individuals, it is important that the website is easy to navigate and quickly find wanted information. In addition, it is important that the website is engaging for younger audiences. To achieve this goal, the Board of Governors website will incorporate links to interactive social media and videos. Staff bios will be recorded and placed on the website to give a face to the Board of Governors office. Also, universities’ promotional videos will be placed on their informational pages to bring life to their campus.
Technology Analysis
Background The technology for this website will be very basic and easy to use. In order to cater to the technological skill of all audience, the website will only require standard knowledge of computer use. This basic design of the website will also assist immediate user familiarity with the site. It is important that navigation and menus are clear and make sense.
ADA Compliance The Americans with Disabilities Act (ADA) requires state and local governments to provide individuals with disabilities equal access to their programs, services, or activities unless doing so would “result in a fundamental alteration in the nature of a service, program, or activity or in undue financial and administrative burdens” (Nondiscrimination on the Basis of Disability in State and Local Government Services, 2010).
As a government organization, the Board of Governors website must be ADA compliant. This requires that the website follows the following accessibility checklist published by the US Department of Health and Human Services:
• A text equivalent for every non-text element shall be provided • Equivalent alternatives for any multimedia presentation shall be
synchronized with the presentation • Web pages shall be designed so that all information conveyed with
color is also available without color, for example from context or markup
• Documents shall be organized so they are readable without requiring an associated style sheet
• Redundant text links shall be provided for each active region of a server-side image map
Team Dartmouth COM5338-01
Usability Testing Report
6
• Client-side image maps shall be provided instead of server-side image maps except where the regions cannot be defined with an available geometric shape
• Row and column headers shall be identified for data tables • Markup shall be used to associate data cells and header cells for data
tables that have two or more logical levels of row or column headers • Frames shall be titled with text that facilitates frame identification and
navigation • Pages shall be designed to avoid causing the screen to flicker with a
frequency greater than 2 Hz and lower than 55 Hz • A text-only page, with equivalent information or functionality, shall be
provided to make a web site comply with the provisions of this part, when compliance cannot be accomplished in any other way; the content of the text-only page shall be updated whenever the primary page changes
• When pages utilize scripting languages to display content, or to create interface elements, the information provided by the script shall be identified with functional text that can be read by assistive technology
• When a web page requires that an applet, plug-in or other application be present on the client system to interpret page content, the page must provide a link to a plug-in or applet that complies with §1194.21(a) through (l)
• When electronic forms are designed to be completed on-line, the form shall allow people using Assistive Technology to access the information, field elements, and functionality required for completion and submission of the form, including all directions and cues
• A method shall be provided that permits users to skip repetitive navigation links
• When a timed response is required, the user shall be alerted and given sufficient time to indicate more time is required (Web-based Intranet and Internet Information and Applications, 2006)
Minimum hardware requirements
PC with Windows 95 or later Mac with OS 9.2 or later
Network connection Screen resolution
o 1024 x 768 minimum Keyboard and mouse
Minimum software requirements
Compatible web browsers: Internet Explorer 6.0 or later Firefox 1.0 or later Netscape 7.2 or later Apple Safari 1.3 or later
Flash Plug-in 6.0 or later
Team Dartmouth COM5338-01
Usability Testing Report
7
Adobe Acrobat Reader 5.0 or above Operating environment
The website will typically be viewed in an office environment. It is likely that users will have few environmental distractions while viewing the Board of Governors website if viewed in the office environment. For users visiting the website at home, the type of information on the website and users’ interest in higher education suggest that users will be focused on navigating the website.
Website Description
Background The Board of Governors website provides information about the Board, the State University System of Florida institutions, and higher education issues in Florida. The website focuses on 6 main areas of content: 1) universities in the System, 2) information about higher education for students, 3) the Board of Governors, 4) in-depth information about universities in the System, 5) press room, and 6) facts and figures about the System.
Project Improvements to the website will incorporate all new endeavors the Board of Governors has undertaken to promote the mission and image of the Board. The newest communications effort has been to promote the Board through social media, such as Facebook and Twitter. The website will include links for users to connect with the Board’s social media pages. The updates will also include a redesigned Frequently Asked Questions (FAQs) page to replace the current one that is difficult to navigate and outdated. The Senior Staff bios will also be updated to include video introductions for each staff member to increase the feeling of accessibility between the public and the Board.
Team Dartmouth COM5338-01
Usability Testing Report
8
PART II: SUPPLEMENTAL DESIGN DOCUMENTS Design Overview/Approach
The redesign of the Board of Governors website takes into account existing color schemes, fonts, images, and design. The purpose of the website redesign is to simplify existing modules and create a structure that is much easier to navigate. In order to accommodate the varying ages and technical capabilities of all primary audience segments, the website design will be simple and easy to use. The website will require only standard knowledge of computer use. Navigation menus will be straightforward and easy to understand. The website will begin to engage younger audiences through links to interactive social media and informational videos. As a government organization, the website must maintain a professional look and sound. Language used in the website will be very straightforward and serve to provide users with information. The website will also follow design guidelines established by the Americans with Disabilities Act (ADA). Colors used in the website will reflect other Board of Governors publications. The current website’s existing color scheme will be slightly updated to a cleaner and sleeker variation. Background and header images will be created to reflect the updated and modernized design. The navigation structure will be flattened so that users can find the information they seek much easier. Fonts that are easy to read will be used in the website.
Global Conventions
In regards to web design, metaphors are used to help guide the user through the navigation of a website and to identify various tasks. These metaphors can be icons or images that mimic real-life objects and associations in order “to bridge some sense of familiarity between virtual and actual life,” (Turnbull, 2011). On the State University System of Florida and the Board of Governors’ website, we will use several metaphors to guide the user. Buttons will alert the user where an action can be made, such as running a search or navigating to a section of the site. Icons will be found around the site to identify key information (e.g. On the Contact Us page, a phone graphic by the phone number and an envelope graphic by the e-mail address). There will be an interactive metaphor in the map of Florida, in which the user can click on stars (representing various universities) located throughout the state and be navigated to pages that provide information on the university.
Team Dartmouth COM5338-01
Usability Testing Report
9
FIGURE 1: Current flowchart.
Publ
ic
Uni
vers
ities
HO
ME
Cho
ose
a U
The
Syst
em*T
he B
oard
Con
tact
Us
*For
Stu
dent
s*F
acts
&
Figu
res
Pres
s Ro
omSe
arch
UW
FEm
ploy
men
tFr
eque
ntly
Re
ques
ted
Publ
ic
Doc
umen
ts
Staf
f Lis
ting
UC
F
21st
Cen
tury
Pr
ogra
ms
FIU
UN
F
Adv
isor
y C
ounc
il of
Fac
ulty
Sen
ates
BOG
Mee
ting
Dat
es
FSU
FAU
Uni
vers
ity
Pres
iden
tsPr
ess
Rele
ase
Arc
hive
s
Goo
gle
Map
s
USF
Self-
insu
ranc
e Pr
ogra
ms
Art
& G
raph
ics
Gal
lery
Park
ing
New
Sc
hool
Prof
essi
onal
Sc
hool
s
2012
-202
5 St
rate
gic
Plan
FGC
UIn
nova
tion
& E
cono
mic
D
evel
opm
ent
New
s C
lips
UF
Facu
lty &
Sta
ff
Sear
ches
Cur
rent
Info
Brie
fs
FPU
FAM
UU
nive
rsity
Boa
rds
of T
rust
ees
New
s
Driv
ing
D
irect
ions
KEY
Loca
tion
of
links
to
page
s
Dro
p-do
wn
men
u
Page
Both
Team Dartmouth COM5338-01
Usability Testing Report
10
FIGURE 2: Current flowchart (For Students).
*For
Stu
dent
s
Kno
w H
ow 2
Go
FAQ
FL D
ept
of
Educ
atio
n O
ffice
of
Equi
ty &
Acc
ess
Plan
ning
for
Col
lege
Aca
dem
ic L
earn
ing
Com
pact
s
Flor
ida
Col
lege
A
cces
s N
etw
ork
Fina
ncia
l Aid
*Tui
tions
& F
ees
Serv
ices
for
Stud
ents
with
D
isab
ilitie
s
Adm
issi
ons
&
Tran
sfer
Info
Uni
vers
ity &
Pr
ogra
m In
fo
Stud
ent
Righ
ts &
Re
spon
sibi
litie
s
FLVC
.org
Ass
ista
nce
for
Solv
ing
Prob
lem
s
Dis
tanc
e Le
arni
ng
Opp
ortu
nitie
s
Und
ergr
adua
te
Adm
issi
ons
OSF
A
FAFS
A
Aca
dem
ic D
egre
e Pr
ogra
ms
Ass
ista
nce
for
Pros
pect
ive
Teac
hers
Gra
nts
for
Firs
t G
ener
atio
n
Hon
ors
Prog
ram
s
& C
olle
ges
Inte
rnat
iona
l St
uden
ts
John
son
Sc
hola
rshi
p
Stud
y A
broa
d Fe
llow
ship
s &
Oth
er
Opp
ortu
nitie
s
Flor
ida
Resi
denc
y Re
quire
men
ts fo
r Tu
ition
Ass
ista
nce
for
Vete
rans
Aca
dem
ic C
omm
on
Mar
ket
Firs
t Ti
me
C
olle
ge S
tude
nts
Aca
dem
ic
Com
petit
iven
ess
Gra
nt
Find
a P
ublic
U
nive
rsity
Tran
sfer
Col
lege
St
uden
ts
Brig
ht F
utur
es
Cos
t of
Att
enda
nce
by U
nive
rsity
Gra
duat
e &
Pr
ofes
sion
al
Adm
issi
ons
Prep
aid
Col
lege
Pr
ogra
m
Team Dartmouth COM5338-01
Usability Testing Report
11
FIGURE 3: Current flowchart (The Board).
*The
Boa
rd
Con
tact
Task
For
ces
BOG
Fou
ndat
ion
Boar
d M
embe
rs
Regu
latio
ns &
G
uide
lines
Phot
o G
alle
ry
Uni
vers
ity B
oard
of
Trus
tees
Staf
f
Cha
ncel
lor
Mee
tings
Budg
et &
Fis
cal
Stra
tegi
c Pl
an
Mis
sion
& V
isio
n
Com
mitt
ees
Mem
ber
Profi
les
Mee
ting
Arc
hive
Mee
ting
Cal
enda
r
Biog
raph
y
Cha
ncel
lor’s
U
pdat
es
Visi
on
Seni
or S
taff
Faci
litie
s
Aca
dem
ic L
ibra
ries
Emer
genc
y M
anag
emen
t
FAM
U F
inan
ce &
O
pera
tion
Acc
ount
abili
ty
Repo
rts
Tuiti
ons
& F
ees
Allo
catio
n Su
mm
ary
Legi
slat
ive
App
ropr
iatio
ns
Aca
d. In
fras
truc
ture
Su
ppor
t O
rgs
Inst
itute
s &
Cen
ters
Bu
dget
Fina
ncia
l St
atem
ents
Uni
vers
ity
Ope
ratin
g Bu
dget
s
Reve
nue
Out
look
[P
DF]
Legi
slat
ive
Budg
et
Requ
ests
Fund
ing
His
tory
Foun
datio
n, In
c.
Expe
nditu
re
Ana
lysi
s
St U
niv
Syst
em F
inal
A
men
dmen
t Pk
gs
Dep
artm
ents
Boar
d Em
ploy
men
t
Staf
f Lis
ting
Team Dartmouth COM5338-01
Usability Testing Report
12
FIGURE 4: Current flowchart (Facts & Figures).
*Fac
ts &
Fi
gure
s
Inte
ract
ive
Uni
vers
ity D
atab
ase
Sust
aina
bilit
y
Qui
ck F
acts
Acr
onym
s
Publ
icat
ions
Budg
et &
Fis
cal
Repo
rts
Glo
ssar
y of
Ter
ms
Stra
tegi
c Pl
an
Acc
ount
abili
ty
Repo
rts
Arc
hive
of
Info
Brie
fs
Cur
rent
Info
Brie
fs
Uni
vers
ity F
act
Book
s
Regu
latio
ns &
G
uide
lines
Forw
ard
by D
esig
n
Expe
rt N
et
Dat
a &
Info
Tec
h Re
sour
ces
Fact
Boo
ks
Educ
atio
n Li
nks
Team Dartmouth COM5338-01
Usability Testing Report
13
FIGURE 5: Redesign flowchart.
Choo
se a
U
HO
ME
Choo
se a
UTh
e Sy
stem
*The
Boa
rdCo
ntac
t Us
*For
Stu
dent
s*F
acts
&
Figu
res
Pres
s Roo
m
UW
F
Freq
uent
ly
Requ
este
d Pu
blic
D
ocum
ents
Sta!
List
ing
UCFFIU
UN
F
Advi
sory
Cou
ncil
of F
acul
ty S
enat
esBO
G M
eetin
g Ca
lend
ar
FSU
FAU
Uni
vers
ity
Pres
iden
t Dire
ctor
y
Pres
s Rel
ease
s
Goo
gle
Map
s
USF
Self
Insu
ranc
e Pr
ogra
ms
Art &
Gra
phic
s G
alle
ry
Park
ing
NCF
Prof
essio
nal
Scho
ols D
irect
ory
Publ
icat
ions
FGCU
Syst
em R
esea
rch
& In
nova
tion
In th
e N
ews
UF
2012
-202
5
Stra
tegi
c Pl
an
FPU
FAM
U
Uni
vers
ity B
oard
s of
Trus
tees
Driv
ing
D
irect
ions
KEY
Loca
tion
of li
nks t
o pa
ges
Expa
ndab
le m
enu
Page
Both
Annu
al
Acco
unta
bilit
y Re
port
Econ
omic
Impa
ct
Repo
rt
Curr
ent I
nfo
Brie
fs
Phot
o G
alle
ry
Exte
rnal
pag
e
BOG
Mee
ting
Arch
ives
Team Dartmouth COM5338-01
Usability Testing Report
14
FIGURE 6: Redesign flowchart (For Students).
*For
Stu
dent
s
Know
How
2 G
o
FAQ
FL D
ept o
f Ed
ucat
ion
O!
ce o
f Eq
uity
& A
cces
s
Acad
emic
Lea
rnin
g Co
mpa
cts
Flor
ida
Colle
ge
Acce
ss N
etw
ork
Fina
ncia
l Aid
Re
sour
ces
*Tui
tions
& F
ees
Serv
ices
for
Stud
ents
with
D
isabi
litie
s
Adm
issio
ns &
Tr
ansf
er In
fo
Uni
vers
ity &
Pr
ogra
m In
fo
Stud
ent R
ight
s &
Resp
onsib
ilitie
s
FLVC
.org
Assis
tanc
e fo
r So
lvin
g Pr
oble
ms
Dist
ance
Lea
rnin
g O
ppor
tuni
ties
Acad
emic
Deg
ree
Prog
ram
s
Hon
ors P
rogr
ams
& Co
llege
s
Stud
y Ab
road
Fe
llow
ship
s & O
ther
O
ppor
tuni
ties
Acad
emic
Com
mon
M
arke
t
*Cho
ose
a U
Cost
of A
tten
danc
e by
Uni
vers
ity
OSF
A
FAFS
A
Assis
tanc
e fo
r Pr
ospe
ctiv
e Te
ache
rs
Gra
nts f
or F
irst
Gen
erat
ion
John
son
Sc
hola
rshi
pAc
adem
ic
Com
petit
iven
ess
Gra
nt
Brig
ht F
utur
es
Prep
aid
Colle
ge
Prog
ram
Fina
ncia
l Aid
at
FLVC
.org
Und
ergr
adua
te
Adm
issio
nsIn
tern
atio
nal
Stud
ents
Flor
ida
Resid
ency
Re
quire
men
ts fo
r Tu
ition
Firs
t Tim
e
Colle
ge S
tude
nts
Tran
sfer
Col
lege
St
uden
ts
Gra
duat
e &
Prof
essio
nal
Adm
issio
ns
Addi
tiona
l Res
ourc
es
Team Dartmouth COM5338-01
Usability Testing Report
15
FIGURE 7: Redesign flowchart (The Board).
*The
Boa
rd
Task
For
ces
BOG
Fou
ndat
ion
Boar
d M
embe
rs
Regu
latio
ns &
G
uide
lines
Sta!
Chan
cello
r
BOG
Mee
ting
Cale
ndar
Budg
et &
Fisc
al
Miss
ion
& Vi
sion
Com
mitt
ees
Mem
ber P
ro"l
es
Mee
ting
Arch
ive
Biog
raph
y
Chan
cello
r’s
Upd
ates
Visio
n
Faci
litie
s
Acad
emic
Lib
rarie
s
Emer
genc
y M
anag
emen
t
FAM
U F
inan
ce &
O
pera
tion
Acco
unta
bilit
y Re
port
s
Tuiti
ons &
Fee
s
Allo
catio
n Su
mm
ary
Legi
slativ
e Ap
prop
riatio
ns
Acad
. Inf
rast
ruct
ure
Supp
ort O
rgs
Inst
itute
s & C
ente
rs
Budg
et
Fina
ncia
l St
atem
ents
Uni
vers
ity
Ope
ratin
g Bu
dget
s
Reve
nue
Out
look
[P
DF]
Legi
slativ
e Bu
dget
Re
ques
ts
Fund
ing
Hist
ory
Foun
datio
n, In
c.
Expe
nditu
re
Anal
ysis
St U
niv
Syst
em F
inal
Am
endm
ent P
kgs
Dep
artm
ents
Boar
d Em
ploy
men
t
Sta!
List
ing
Seni
or S
ta!
Boar
d O
rgan
izat
iona
l Ch
art
Team Dartmouth COM5338-01
Usability Testing Report
16
FIGURE 8: Redesign flowchart (Facts & Figures).
*Fac
ts &
Fi
gure
s
Inte
ract
ive
Uni
vers
ity D
atab
ase
Qui
ck F
acts
Publ
icat
ions
Budg
et &
Fisc
al
Repo
rts
Glo
ssar
y of
Term
s
Stra
tegi
c Pl
an
Acco
unta
bilit
y Re
port
s
Arch
ive
of
Info
Brie
fs
Curr
ent I
nfo
Brie
fs
Uni
vers
ity F
act
Book
s
Regu
latio
ns &
G
uide
lines
Forw
ard
by D
esig
n
Dat
a &
Info
Tech
Re
sour
ces
Fact
Boo
ks
Team Dartmouth COM5338-01
Usability Testing Report
17
FIGURE 9: Storyboard (Home page).
Team Dartmouth COM5338-01
Usability Testing Report
18
FIGURE 10: Storyboard (For Students).
Search
LOGO
Choose a U
STATE UNIVERSITY SYSTEM of FLORIDABoard of Governors
For Students The System Facts/FiguresThe Board Press Room Contact Us
Copyright 2012 Saeperia nient lam faceperis debit, te suntis simus sit prorror itasimet molorectio venem nis sint ant la dem conse volor autatem. Into quidundunt licimolupta sed quam re odis dendiatur, net maio officia ium repres ipsam que pores si odit, eum audandi onsenis ciducip ica
Pudae il ipsam et aspiciis dolo vent laborer iorrum re cus alictur?Uptat estia voluptatias ut autem etur? Quibustem vel id ulpa in consenihil ma sandic te con rehent, sed magnis sectatet moluptaquo eatur?Catiis esed qui is sum vel im ipis unt et quisimint quam, tetur, velecepero eum, core re nonsequ ossequibusda cus, incia sitaes enis eum rerist veliquae lam voluptatem aut ipsa prae vellent.Ratioris et od ex eventusae paris et untis ped eum rest preptat.Ene volore, iusa sint maximil laccus dis audaeped molore milic te commodios sandam que pora et as doluptae vit offic tem lam a con consectis eaquia volorum dest quis con con postiunte pre rehendis intis doluptat dunto cor audant adis magnihicium que et odit, veles il im quidus deratemqui dolesci aerovit que vid quiam, aut porit ad mi, cuscia doluptur aut verroreped magni doluptatur assimiliciam audipsa picipid undita sunt qui ni omnimporiam, ut qui dernam et il inum inctem quias minimpo stisitint aceribus, ullupit experum eum eatiaturis derestia evenihil modit quae. Nem quidipsuntia qui non conse-quam acea que non earunti orenientia simi, occatur?Sust, a vellenditis in coreius temquat qui rerunt velendunt porepudaes velist, corpos dolest hariatectius simi, torere voluptatus aligend igendebit utem iusapel luptat.Agnatur, conse officimusa quis vita diciis culla quid ut la digeni dolorer ferspit offic te ne re num faccatium volum qui tor re si quatia dipsum a dolestiis-tiaOtam hos, C. Opiocum ad confintem oma, silin norsultus, facii pra ete imis ius renatqua nora re et rem liam in re acernitusus, esterrio vil tra musquam ina at nere egeres consul hostori sendierte antillesidem dellarbit, Catimil iquamqu ondepon timprat. Catus pliam re, sendicasdam spio, tuam nihilne mpopte, non vid caeciam iusse intemquodius in det victus, qua vit, que im-ortem, est? At inatrae auc viu in se fatum, consupicas nit; nostrac eripioreni senatil iceperes iam. Arem virit vis. Consima ximihic aequidiere terem te hu-con di senterus virit, fuis, quit re, Catu es etestorsulus hum horipicae conitem publissimis mena, concerceperi tem locasdam omnonvest abi pultus, num et grachuit. con Etra? que maximus con tes sticaecrec vesimusque nonsua vius, dero ertiam interis, vir in hocus tui se mo horemurnica; nonessis. Obsenatis
Home > For Students
Itam pon hali pulus auciaequam iamena, telabus hosul habus con
clum, Cuperum menatus, condum sendier adet in ium. Gra me ad
incenatus bon telabunclus, coendam
dius, de inam nihintem pravocc ibus-qua veridie mursullestis aude
hos inatifex nosulab essimen icatis
o iactum simissi mperem vo, ne tus conequo nfenihilius veres ad consu-
FOR STUDENTS
QUICK LINKS
Assistance for Solving Problems
Student Rights and Responsibilities
Admissions and Transfer Information
Academic Learning Compacts
Planning for College
FAQ
Tuition and Fees
University and Program Information
Services for Students with Disabilities
Financial Aid
Team Dartmouth COM5338-01
Usability Testing Report
19
FIGURE 11: Storyboard (Sidebar expanded menu).
Search
Pudae il ipsam et aspiciis dolo vent laborer iorrum re cus alictur?Uptat estia voluptatias ut autem etur? Quibustem vel id ulpa in consenihil ma sandic te con rehent, sed magnis sectatet moluptaquo eatur?Catiis esed qui is sum vel im ipis unt et quisimint quam, tetur, velecepero eum, core re nonsequ ossequibusda cus, incia sitaes enis eum rerist veliquae lam voluptatem aut ipsa prae vellent.Ratioris et od ex eventusae paris et untis ped eum rest preptat.Ene volore, iusa sint maximil laccus dis audaeped molore milic te commodios sandam que pora et as doluptae vit offic tem lam a con consectis eaquia volorum dest quis con con postiunte pre rehendis intis doluptat dunto cor audant adis magnihicium que et odit, veles il im quidus deratemqui dolesci aerovit que vid quiam, aut porit ad mi, cuscia doluptur aut verroreped magni doluptatur assimiliciam audipsa picipid undita sunt qui ni omnimporiam, ut qui dernam et il inum inctem quias minimpo stisitint aceribus, ullupit experum eum eatiaturis derestia evenihil modit quae. Nem quidipsuntia qui non conse-quam acea que non earunti orenientia simi, occatur?Sust, a vellenditis in coreius temquat qui rerunt velendunt porepudaes velist, corpos dolest hariatectius simi, torere voluptatus aligend igendebit utem iusapel luptat.Agnatur, conse officimusa quis vita diciis culla quid ut la digeni dolorer ferspit offic te ne re num faccatium volum qui tor re si quatia dipsum a dolestiis-tiaOtam hos, C. Opiocum ad confintem oma, silin norsultus, facii pra ete imis ius renatqua nora re et rem liam in re acernitusus, esterrio vil tra musquam ina at nere egeres consul hostori sendierte antillesidem dellarbit, Catimil iquamqu ondepon timprat. Catus pliam re, sendicasdam spio, tuam nihilne mpopte, non vid caeciam iusse intemquodius in det victus, qua vit, que im-ortem, est? At inatrae auc viu in se fatum, consupicas nit; nostrac eripioreni senatil iceperes iam. Arem virit vis. Consima ximihic aequidiere terem te hu-con di senterus virit, fuis, quit re, Catu es etestorsulus hum horipicae conitem publissimis mena, concerceperi tem locasdam omnonvest abi pultus, num et grachuit. con Etra? que maximus con tes sticaecrec vesimusque nonsua vius, dero ertiam interis, vir in hocus tui se mo horemurnica; nonessis. Obsenatis
Home > For Students > Transfer Students
TRANSFER STUDENTS
LOGO
Choose a U
STATE UNIVERSITY SYSTEM of FLORIDABoard of Governors
For Students The System Facts/FiguresThe Board Press Room Contact Us
Copyright 2012 Saeperia nient lam faceperis debit, te suntis simus sit prorror itasimet molorectio venem nis sint ant la dem conse volor autatem. Into quidundunt licimolupta sed quam re odis dendiatur, net maio officia ium repres ipsam que pores si odit, eum audandi onsenis ciducip ica
Undergraduate Admissions
Graduate and Professional Admissions
Transfer Students
First Time College Students
Itam pon hali pulus auciaequam iamena, telabus hosul habus con
clum, Cuperum menatus, condum sendier adet in ium. Gra me ad
incenatus bon telabunclus, coendam
dius, de inam nihintem pravocc ibus-qua veridie mursullestis aude
hos inatifex nosulab essimen icatis
o iactum simissi mperem vo, ne tus conequo nfenihilius veres ad consu-
QUICK LINKS
Assistance for Solving Problems
Admissions and Transfer Information
Planning for College
University and Program Information
Student Rights and Responsibilities
Academic Learning Compacts
FAQ
Tuition and Fees
Services for Students with Disabilities
Financial Aid
Team Dartmouth COM5338-01
Usability Testing Report
20
FIGURE 12: Storyboard (Choose a U).
Search
LOGO
Choose a U
STATE UNIVERSITY SYSTEM of FLORIDABoard of Governors
For Students The System Facts/FiguresThe Board Press Room Contact Us
INTERACTIVE MAP OF FLORIDA
Copyright 2012 Saeperia nient lam faceperis debit, te suntis simus sit prorror itasimet molorectio venem nis sint ant la dem conse volor autatem. Into quidundunt licimolupta sed quam re odis dendiatur, net maio officia ium repres ipsam que pores si odit, eum audandi onsenis ciducip ica
CHOOSE A U
Vertus acidientem poris, curobse non tatuam ne movignonsua mentrari, vivivem, pere, quam
Vertus acidientem poris, curobse non tatuam ne movignonsua mentrari, vivivem, pere, quam
Vertus acidientem poris, curobse non tatuam ne movignonsua mentrari, vivivem, pere, quam
Vertus acidientem poris, curobse non tatuam ne movignonsua mentrari, vivivem, pere, quam
Vertus acidientem poris, curobse non tatuam ne movignonsua mentrari, vivivem, pere, quam
Vertus acidientem poris, curobse non tatuam ne movignonsua mentrari, vivivem, pere, quam
Vertus acidientem poris, curobse non tatuam ne movignonsua mentrari, vivivem, pere, quam
Vertus acidientem poris, curobse non tatuam ne movignonsua mentrari, vivivem, pere, quam
Vertus acidientem poris, curobse non tatuam ne movignonsua mentrari, vivivem, pere, quam
Vertus acidientem poris, curobse non tatuam ne movignonsua mentrari, vivivem, pere, quam
Vertus acidientem poris, curobse non tatuam ne movignonsua mentrari, vivivem, pere, quam
Vertus acidientem poris, curobse non tatuam ne movignonsua mentrari, vivivem, pere, quam
Team Dartmouth COM5338-01
Usability Testing Report
21
Navigation Table Action Item Action Description Comments Navigation menu item button
Click on desired navigation menu item button Leads to the page that corresponds with the item
Two button states (resting, and hover/down)
Search data entry field (text) activated by magnifying glass button
Types in search term, clicks magnifying glass button Leads to a display with clickable results list or “no results” dialog
Two button states (restings/hover, and down)
The search engine uses a string matching algorithm to approximate misspelling. User navigates to search returns
Typeface/Font Specification Usage Typeface Font Example Headers Avenir LT Std 95 Black,
12pt Lorem ipsum dolor sit amet
Navigation menu Avenir LT Std 65 Medium, 10pt
Avenir LT Std 95 Black, 10pt
Banner description Avenir LT Std 45 Book, 12pt
Lorem ipsum dolor sit amet
Body text Garamond Regular, 10pt Lorem ipsum dolor sit amet
Color Table Layout item Example RGB Value Navigation menu, footer, header backgrounds
Navy: 6, 59, 91
Sidebar/menu background
Light Gold: 234, 217, 182
Team Dartmouth COM5338-01
Usability Testing Report
22
Sidebar/menu header backgrounds
Dark Gold: 208, 182, 137
Body text, Sidebar/menu headers
Black: 0, 0, 0
Body background, headers, navigation menu
White: 255, 255, 255
Image Table Layout Item Example Logo banner
Background image
Button States Button Resting Over Down Navigation menu item Search
Team Dartmouth COM5338-01
Usability Testing Report
23
FIGURE 13: Design Comp (Home Page).
Team Dartmouth COM5338-01
Usability Testing Report
24
FIGURE 14: Design Comp (Choose a U).
Team Dartmouth COM5338-01
Usability Testing Report
25
PART III: USABILITY TESTS & RESULTS
Overview The purpose of the test was to assess the usability of the Board of Governors website for users. Specific tasks were identified that utilized different areas of the website’s navigation and tested users’ ability to understand the website’s structure and navigation. Method
The test was carried out on 5 participants who all belong to either a primary or second audience segment for the website. Participants were tested individually. Project team members acted as test facilitators, either individually or in pairs. Each test took between 10 and 15 minutes. The usability test consisted of five parts: initial briefing, pre-test-interview, solving test tasks, post-test interview and questionnaire. Briefing: Test facilitator informs participants about the process
and encourages them to think aloud. Participants sign an agreement allowing facilitators to record the testing.
Pre-test interview: Test participants are asked a few brief questions about their typical Internet use.
Solving test tasks: Test participants are asked to complete tasks using the website.
Tasks are predefined by the test facilitators and described to the participants.
Test participants are encouraged to think aloud and comment on areas of the website they think could be improved.
Post-test interview: Test participants are asked a series of questions about their experience using the website.
Questionnaire: Test participants are asked to fill in a questionnaire while
thinking aloud about the choices they select. The questionnaire asks about the participant’s experience testing the website as well as demographic information.
Equipment
The equipment used for this test was a paper prototype of the website homepage and pages associated with test tasks. Web pages, sidebar menus and drop down menus were part of the paper prototype and able to be implemented as a test participant “clicked” through the site.
Team Dartmouth COM5338-01
Usability Testing Report
26
The paper prototype was laminated so as to be more durable and withstand testing. Audio recording was done on an iPhone.
Test Participant Profiles Participants were tested from two primary audience segments: 1) prospective/current students of higher education and 2) Board of Governors/Boards of Trustees members and staff.
Participant Gender Age
Range Occupation
Audience Segment
Level of computer
skill*
Familiar with the website before
testing?
1 Male 18-29 Graduate Student
Prospective/ current
students of higher
education
Intermediate No
2 Female 18-29 Graduate Student
Prospective/ current
students of higher
education
Proficient No
3 Female 18-29 Undergraduate
Student/ Designer
Prospective/ current
students of higher
education
Expert No
4 Female 18-29 Undergraduate
Student/ Graphic Designer
Prospective/ current
students of higher
education
Proficient No
5 Female 18-29
Director of Communications,
Board of Governors
Board of Governors/ Boards of Trustees
members and staff
Proficient Yes
*Computer skill was ranked by the test participant on the following scale: 1. Expert 2. Proficient 3. Intermediate 4. Basic 5. None
Team Dartmouth COM5338-01
Usability Testing Report
27
Findings While all of the usability test participants were able to complete the tasks, they all also experienced problems with one or more of the tasks at hand. These results show that while the new site design is an improvement, there are still changes that could be made in order to improve the website usability. The following sections outline in detail the specific issues and strong points that were discovered during testing.
Task Results The following chart lists all test tasks and participants. Each entry indicates how successful the participant was in solving that particular task.
✔ Solved without any problems ✔ Solved with minor problems ✔ Solved, but serious problems delayed the participant ✖ Participant was not able to solve the task
Participant 1
Participant 2
Participant 3
Participant 4
Participant 5
Find Kim Wilmath info ✔ ✔ ✔ ✔ ✔
Find enrollment info ✔ ✔ ✔ ✔ ✔
Find FSU info ✔ ✔ ✔ ✔ ✔
Design Overall, all of the participants found the visual design of the website to be pleasing. They found the colors and the layout of the website to be appropriate and professional for the Board of Governors. One of the participants said while it was appropriate, the design was not particularly interesting. Additionally, most participants had difficulties identifying the purpose of the site and overlooked the title.
Recommendations: The title of the site (The State University System Board of Governors) should be made larger so that the user can immediately identify the purpose of the website. Also, the home page should tell the user what the Board of Governors does with a description in the body.
Team Dartmouth COM5338-01
Usability Testing Report
28
Navigation Issues of navigation were present in tests with every participant. Those who were aware of the Board of Governors and the website prior to the test had an easier time in general navigating through the tasks.
One of the major issues discovered during our usability test was the prominence of the navigation menu at the top of the page. Two of our five participants did not immediately recognize the menu and instead looked elsewhere for how to navigate the site. They found the body and the various sections of the body of the page to be more attractive than the navigation bar. All of the participants found issues with navigation of one or more of the tasks. During the first task, participants displayed some confusion with which page to visit in order to find information about the staff member. The most frequent issue was choosing Board Members from The Board page over Staff, as was chosen by three out of five participants did. This issue could have stemmed more from the lack of knowledge about the Board of Governors and its structure. One participant also chose Contact and Press Room, both unanticipated choices, before finding the proper route. The second task was the most complicated task and presented the most problems. The first issue was navigating to the Facts/Figures page, which was the necessary first step toward finding enrollment information. Most participants either tried to navigate to the general university page or showed a delaying confusion over where to begin. Once on the page, the participants voiced an overwhelming feeling over the amount of information on the page. The anticipated route was to choose the Interactive University Database, which easily led the participant to find Enrollment information once they were there. Unpredictably, most participants went searching through the vast number of links on the page. The final task of navigating to the Florida State University page showed the least number of problems. Two unexpected options were presented – participants clicked the FSU logo in the footer to take them to the FSU website and also selected The System.
Recommendations: The navigation bar at the top of the page should be made larger and more prominent. The amount of content contained on the main Facts/Figures page should be organized in a way that it doesn’t seem overwhelming to the user, perhaps by adding more white space. The main page of Facts/Figures should also explain the purpose of the links from the sidebar that is not immediately apparent. The logos at the bottom and The System page should have been able to take the user to find more information about the university, but we were
Team Dartmouth COM5338-01
Usability Testing Report
29
unprepared for these during the usability test. These options will be links in the actual website.
Other
A recurring comment was the use of “Choose a U.” Participants found that the use of “U” instead of “University” was immature and inappropriate for use on a professional website. The amount of content on the website was overwhelming to the user and made navigation difficult. Presenting the user with so many options created confusion over where they should go.
Recommendations: “U” should be changed to “University.” All of the information on website is important in order to satisfy the needs of all of the Board of Governor’s audience segments. While we cannot remove any information, the presentation can be simplified and made more manageable for the viewer.
Usability Test Script
Test Facilitator Script Hi, ______________. My name is Lenore/Whitney, and I will be walking you through this testing session today. Before we begin, I have some information for you, and I am going to read it to make sure that I cover everything. We have asked you here to test a Web site that is being developed so that we can see whether it works as intended. This session should only take about 15 minutes. I want to make it clear that we are testing the site, not you. You can’t do anything wrong here so please don’t worry about making mistakes. Also, please don’t worry about hurting our feelings. We want to improve the site so we need your honest opinions. As you use the site, I’m going to ask you as much as possible to think out loud. Say what you are looking at, what you are trying to do, what you are about to click on, and what you’re thinking. This will greatly help us. If you have any questions as we go along, please ask them although I am going to warn you that I may not be able to answer them. We are interested in learning how people manage the site when they don’t have someone helping them. I’m going to ask you to sign a simple permission form for us and then we’ll begin.
Team Dartmouth COM5338-01
Usability Testing Report
30
Usability Test Consent Form [See Figure 1]
Pre-Test Interview
Any questions so far? Okay, just a few quick questions:
o How many hours a week (just an estimate) do you spend using the Internet at work and at home (including Web browsing and email)?
o What kinds of sites are you looking at when you browse the Web? o What is your interest level in higher education? o Where do you typically get your news (newspaper, TV, word of mouth,
blog, etc)? Test Tasks
Familiarization Take a moment to look at the homepage and familiarize yourself with the page.
What is your overall impression of the page and its colors? What do you think of the layout? Based on the content of the homepage, what do you think the company does?
Tasks Task 1
You are looking for information about a person who works at the Board of Governors. Navigate to the page where you can find information about Kim Wilmath, Director of Communications. Navigate back to the homepage.
Task 2 You are trying to find enrollment statistics for a university in the State University System. Find enrollment information from the last 10 years for the Comm and Media Studies program at Florida State University. Navigate back to the homepage.
Task 3 You are considering applying to Florida State University and want to know more about the university. Navigate to the page where you can find more information about FSU.
Team Dartmouth COM5338-01
Usability Testing Report
31
Post-Test Interview Is the content of the Florida Board of Governors website relevant to you?
Is there anything missing?
Do you think anything should be presented differently?
Did you find the content easy to find?
Was the website organized in a way that made sense to you?
Do you have any comments regarding the colors and/or graphics on the website?
If you were to identify two things you found especially good about the website, what would they be?
If you could give the Florida Board of Governors two pieces of advice, what would those be?
Will you use this website in the future?
Questionnaire
I have brought a brief questionnaire that I would like you to fill out. Please continue to think aloud as you make your selections so I understand why you agree or disagree with the statement.
[See Table 1]
Team Dartmouth COM5338-01
Usability Testing Report
32
FIGURE 1: Consent Form
USABILITY TEST CONSENT FORM State University System of Florida, Board of Governors
Please read and sign this form. In this usability test:
• You will be asked to perform certain tasks on a website. • We will also conduct an interview with you. • You will be asked to fill in a questionnaire.
Participation in this usability study is voluntary. All information will remain strictly confidential. The descriptions and findings may be used to help improve the web site. However, at no time will your name or any other identification be used. You can withdraw your consent to the experiment and stop participation at any time.
A usability consultant will record your presentation, interview, or notes. This recording will be used in review for reporting of the usability test results. By signing this release, you give us the right to record your presentation, interview, or notes for academic purposes at Florida State University and to include your interview in our usability report. No parties will have access to the recordings outside of the usability consultants administering the usability test and the overseeing instructor (Dr. Jonathan Adams).
If you have any questions after today, please contact Lenore Messler at (954)240-3127 or Whitney Payne at (352)206-5031. By signing this document, I have read and understood the information on this form and had all of my questions answered. I give permission to record my participation in this usability test and for use in the usability test report. ______________________________________ _________________ Subject's Signature Date ______________________________________ _________________ Usability Consultant Date
Team Dartmouth COM5338-01
Usability Testing Report
33
TABLE 1: Post-Test Questionnaire
Assessment of the Florida Board of Governors website Mark your satisfaction with the website you have just worked with by circling the figure that reflects your opinion. Fully
agree Disagree
entirely
1. It is difficult to find the information I want 5 4 3 2 1
2. There is nothing superfluous on the website 5 4 3 2 1
3. I was able to quickly find what I want on the website
5 4 3 2 1
4. The website has a logical structure 5 4 3 2 1
5. I have a good understanding of the website’s structure
5 4 3 2 1
6. The website helps me find what I am looking for 5 4 3 2 1
7. I do not like to use the website 5 4 3 2 1
8. I feel efficient working with the website 5 4 3 2 1
9. I was familiar with the website before testing 5 4 3 2 1
Circle the appropriate response or fill in the blank to the following demographic questions. What is your gender? Male Female
What is your age? 18-29 30-39 40-49 50-59 60+
What would you rank your level of computer skill?
Expert Proficient Intermediate Basic None
What is your occupation?
Team Dartmouth COM5338-01
Usability Testing Report
34
REFERENCES Board of Governors, (2008). Mission and Vision. Retrieved from website:
http://flbog.edu/_doc/BOG_MISSION.pdf Nondiscrimination on the Basis of Disability in State and Local Government Services,
28 C.F.R. §§ 35.149, 2 35.164 (2010). Web-based Intranet and Internet Information and Applications, 36 C.F.R. §1194.22
(2006). Turnbull, Connor (2011), Using Metaphors in Web Design. Retrieved from website:
http://webdesign.tutsplus.com/articles/design-theory/using-metaphors-in-web-design/