EDF Information Center

Usability Test Results and Recommendations

November 2007

Scope of testing

Usability of the interface Content (presence, not quality of info) Home page organization & navigation Search feature Toolbar actions Glossary access

Most of STA-SI users In-depth testers = 9 Brief testers = 18 Total = 27 of 36 target users, or 75% Novice, intermediate, and expert EDF users

Quotable quotes

I can see it still needs work, but how do I rate the idea of it? It’s out of this world!

This is something that should have happened a long time ago.

I can see it to be very useful—a one-stop shop.

Can I bookmark this site? [Yes!]

Rating the overall help system

Data from 9 in-depth testers4= Excellent 3= Good 2= Fair 1= Poor

“It’s like dating.” (A few excellent ratings dropped to good.)

3.22 3

1

2

3

4

Pre-test rating Post-test rating

Rating the overall experience

Data from 9 in-depth testers3= Fully sat 2= Somewhat sat 1= Unsatisfactory

Post-test ratings

Using the Search feature

2

1

2

3

Looking for information

1.89

1

2

3

Users’ top information needs

Data processing, 18

Metadata, 17

Retrieve & compare, 9

EDF general, 7

Data management, 6

Publications & containers, 6

EDFExcel, 5

Other, 5

Top information needs found?

Results from all 27 users Chose 1 of 3 top needs

Yes43%

No57%

Comparative test results (1 of 3)

Completion rate

91%

44% 39%

96%

0%

20%

40%

60%

80%

100%

Home Page Search Toolbar Glossary

Comparative test results (2 of 3)

Time to complete(estimated max minutes)

1.68

2.37

1.22 1.11

1.00

2.00

3.00

Home Page Search Toolbar Glossary

Comparative test results (3 of 3)

Number of false attempts

0.72

2.30

0.81

0.17

0.00

0.50

1.00

1.50

2.00

2.50

Home Page Search Toolbar Glossary

Results interpreted (1 of 2)

Home page navigation moderately difficult users studied the page a long time before clicking data

usually led to the answer (91% completion) was fairly time-consuming (<1.68 minutes) with few errors (.72 false attempts)

Search very difficult users tried several searches users studied results lists, often without clicking any items data

often did not lead to answer (44% completion) was very time-consuming (<2.37 minutes) with moderate errors (2.30 false attempts)

Results interpreted (2 of 2)

Toolbar not noticeable after error, users typically stopped trying or didn’t

notice their error data

usually was missed (39% completion) spent little time (<1.22 minutes) with few errors (.81 false attempts)

Glossary easy data

usually led to the answer (96% completion) was quick (<1.11 minutes) with few errors (.17 false attempts)

Users’ top comments & suggestions

Bland appearance; columns out of whack Overwhelming amount of content on home page Ambiguous major headings Links on the main page are not descriptive Search functionality is not intuitive Unique country processing app. needs to go live Layout of document links & abstracts is confusing Toolbar isn’t noticeable; also needs better tips Left pane is in the way; doesn’t help Country codes need link to db for accuracy Users need to be able to add/modify content

Next steps (1 of 3)

Improve appearance & organization: Simplify the home page Make the toolbar larger with better tool

tips; include all tools needed to avoid alternating w/browser toolbar

Redesign the page layout Eliminate drop-down abstracts for

documents; use “summary” instead of “abstract” if needed

Next steps (2 of 3)

Improve content retrieval: Convert long documents to help topic

chunks for easier search, retrieve, scan, & use

Modify the Search feature’s method of operation (if possible) or provide search tips

Next steps (3 of 3)

Enhance technology: Link to country name database Finish the country processing app. Enable user content contribution