Presented December 7, 2006 on the campus of Southern Polytechnic State UniversityMarietta, Georgia

Usability Test Results forwww.southernartistry.org/artistcenter

Sponsor: Hannah Leatherbury, Southern Arts Federation

Team Members: Marty Martin Travis Melton Lori Rich Edward Simpson

Usability Study Report Fall 2006www.southernartistry.org Southern Polytechnic State University

Table of Contents

Executive Summary........................................................................................... ivIntroduction.........................................................................................................1

Overview of the Artist Center Web Tool.........................................................................1Test Background............................................................................................................2Test Purpose and Scope................................................................................................2Test Objectives...............................................................................................................3Report Format................................................................................................................4

User Selection Methodology and Profiles.........................................................6Personas........................................................................................................................6Novice Users..................................................................................................................6Intermediate/Advanced Users........................................................................................7Recruitment and Screening............................................................................................7

Methodology: Description of the testing process..........................................10Planning Phase............................................................................................................10

Heuristic Evaluation..................................................................................................10Design of Questionnaires.........................................................................................12Description of the Scenarios.....................................................................................12

Scheduling....................................................................................................................15Testing Environment & Equipment...............................................................................15Roles of the Team........................................................................................................16

Findings.............................................................................................................17User 7.......................................................................................................................17

Quantitative Findings...................................................................................................17Qualitative Findings......................................................................................................23Post-Task Responses..................................................................................................26

Conclusions and Recommendations..............................................................31Summary of positive findings.......................................................................................31Bug Fixes.....................................................................................................................31Areas of highest usability concern...............................................................................32Feature Enhancements................................................................................................33

AppendicesAppendix A - Heuristic Evaluation………………………………………………….1Appendix B - Screener Questionnaire.............................................................21Appendix C - Pre-Test Questionnaires............................................................22Appendix D - Test Scenarios............................................................................33Appendix E - Post-Task Questionnaires.........................................................37Appendix F - Post-Test Questionnaires..........................................................57Appendix G - Facilitator Script.........................................................................64Appendix H - Testing Session Logs (Morae).................................................................66

ii

Usability Study Report Fall 2006www.southernartistry.org Southern Polytechnic State University

List of IllustrationsTablesTable 1: Test Participants’ Characteristics……………………………………………………9Table 2: Nielsen’s Ten Heuristics…………………………………………………..……......10Table 3: Team Roles and Responsibilities………………………………………………….17Table 4: Success and Time Performance……………………………………………..…....18Table 5: Test Log Summary……………………………………………………………….….19Table 6: Scenario 1 Post-Task Responses……………………………………………..…..26Table 7: Scenario 2 Post-Task Responses…………………………………………….…...27Table 8: Scenario 3 Post-Task Responses…………………………………………………28Table 9: Scenario 4 Post-Task Responses…….. ………………………………………….29Table 10: Post-Test Responses……………………………………………………………...30Table 11: Areas for usability improvements…………………………………………………32

FiguresFigure 1: Artist Center Home page………………………………………………………….…2

Figure 2: View of the usability lab…………………………………………………………….15

iii

Executive Summary

This report presents the results from a usability study of the Artist Center interface conducted over a three week period during the months of November and December 2006.

Six test participants who mirrored real-world users were selected to participate in the usability study. The participants were from different artistic backgrounds and had varying levels of computer and internet skills. Most were novice users in respect to the Artist Center interface, while some of the participants had existing artist profiles.

The usability test was based on a series of tasks that required the test participants to create or edit an artist profile and upload work samples. The goal of the usability test was to quantify the usability factors at play as a user creates his or her portfolio. The test results provide concrete data to support any design improvements for the tool.

The most significant findings from the study include the following:

Significant Findings Recommended Fix

Excessive scrolling – Half of the participants had difficulty navigating the tool due to issues with excessive scrolling.

Eliminate all scrolling throughout the page.

Move instructions to a popup window, and use “Next” and “Back” buttons between each field instead of requiring scrolling.

Inadequate error prevention – 4 out of 6 participants had difficulty differentiating uploading work sample tasks versus inputting descriptive content.

Reorder the tasks – Have the user upload the file prior to entering descriptions.

Provide a step-by-step list of tasks to be completed for uploading work samples.

Inadequate error handling – Half of the participants experienced issues with error messages.

Highlight actual errors on screen. Move error message next to the error.

Confusing/Inflexible form behaviors - Errors on forms clear all fields, not just the field in error.

Highlight form input error without clearing all fields.

Violation of web navigation standards – All of the test participants attempted to access tab graphics with the assumption that they were links instead of tabs.

Add link functionality to the tab graphics.

Distinguish headers from links. Open FAQ in a new, reduced-size

window.Visibility of system status – A majority of the participants had issues with the “Generic Icon” option due to it not being clearly marked.

Have system default to generic icon. Shorten and clarify the instructions for

the thumbnail image.Recognition rather than recall – Half of the test participants did not successfully upload work samples due to a number of usability issues.

Create a separate input form for representative’s information with clear headers.

Introduction

Overview of the Artist Center Web Tool SouthernArtistry.org provides artists and arts groups with an online platform from which to expand their audience, publicize their work, and articulate their artistic philosophy. (Source: http://www.southernartistry.org)

Invited artists can use their completed portfolios to market their work through the following functions:

Online Profile (bio/group history; awards; services; reviews) Work Samples (images; writing; music tracks; video) Publications Lists (Books and articles) Upcoming Events Calendar (Tours; readings; lectures; exhibits; festivals) Links to Additional Websites that showcase the artist’s work

This usability test report focuses on the artist center interface or web tool, which serves as an online tool for the creation of a web-based portfolio. The interface requires a logon ID, which is provided by the Southern Arts Federation to the artist once he or she is chosen to be featured on the site.

Figure 1: Artist Center Home Page

Test BackgroundThis usability study was completed on the campus of Southern Polytechnic State University in Marietta, Georgia during the months of November and December 2006. Under the guidance and direction of Dr. Carol Barnum, the team members and the sponsor recruited volunteer test participants, and then screened each user to ensure that he/she truly represented the interests and abilities of the artists who are profiled on SouthernArtistry.org.

The testing process lasted approximately 50-55 minutes per user and directed the test participant to complete four primary scenarios involved in building or updating an artist portfolio. During the testing sessions, the team observed and recorded any difficulties and/or successes the users encountered. The test participants were given a small stipend for their participation.

Upon completion of the testing sessions, the team analyzed the test data, which served as source data for this report and a presentation for the sponsor.

Test Purpose and Scope The usability test for the artist interface was chartered by the team members and the test sponsor with the following purposes:

1. To document any issues with the tool that may be causing frustration for the artists and extra work for the personnel who support the website

2. To recommend fixes for those issues that create frustration for or generate errors from the users

3. To document any positive usability elements found on the site, so that the tool owners will further capitalize on these elements in the next redesign

4. To educate the team members on the processes, procedures, and value of usability testing

The content / scenarios for the test were limited to the following four scenarios that must be completed to build an artist portfolio:

1. Understanding the basic navigation and functions of the tool

2. Building an artist profile

3. Uploading work samples

4. Updating the completed artist portfolio

Test ObjectivesThe objectives of the test were to quantify the usability factors at play as a user creates his or her portfolio. The test results provide concrete data to support any design improvements for the tool. Qualitative answers to these questions are provided in the “Findings” section of this report.

Among the questions the study answers are these:

Can the user navigate the screens easily throughout the portfolio-creation process? How often does he/she hesitate or become confused?

Which tasks/subtasks cause the most difficulty for the users?

Can the user easily complete tasks not directly related to uploading content, such as creating keywords to make the profile searchable or adding a “remove date” to content?

How frequently does the user need to call the help desk and/or consult the FAQ for assistance? Does the FAQ provide the answer quickly and easily?

Does the user understand what he/she is doing and why? Or is he/she just following instructions? Does the system provide enough feedback to the user?

How does a new user interact with the site? Does he/she experience frustration or difficulty in completing the four tasks required to build a portfolio?

What are the user’s opinions of the site after building a portfolio versus prior to attempting the tasks?

Does the user believe he/she needs to read through the printed introductory materials prior to building a portfolio?

Report Format

1. User Selection Methodology and Profiles The project team developed user personas prior to recruiting any test participants in order to screen and select users who accurately reflect the real-world users of the site. These personas are described here. Section two also describes the participant recruitment and screening process. Finally, short descriptions of each of the test participants are included.

2. Test MethodologyThe methodology section outlines the steps followed in planning and administering the test. It also describes how the team compiled and analyzed the data from the completed testing sessions.

3. Findings and RecommendationsThe results of the tests are described in this section. Both positive and negative findings are detailed with supporting data from the testing sessions. Where usability deficiencies are found, recommendations for design improvements are included.

4. AppendicesThe appendices are a collection of the various documents created and used during the testing process. They include the following documents or categories of documents:

a. Heuristic ReviewAn illustrated, pre-test usability review of the site completed by the project team members

b. Screener QuestionnaireThe questionnaire completed by each participant to determine if they fitted our user profile

c. Pre-Test Questionnaires The questionnaires completed by each test participant immediately before the testing session

d. Test ScenariosThe scenarios or tasks that each user attempted or completed during the testing session

e. Post Task Questionnaires The questionnaires completed by each user after each scenario

f. Post-Test Questionnaire The questionnaires complete by each user after completing the testing session

g. Facilitator ScriptThe checklist or script used by the test facilitator to administer the test

h. Session Logs (Morae)The written logs of narration provided by the observing team members during the test

User Selection Methodology and Profiles

PersonasThree user personas were generated prior to recruiting test participants. This was done in an effort to create guidelines to screen and select test participants that mirrored the interfaces’ real-world user. Each of the three personas represented a different aspect of the user. This criterion was generated from the user research conducted during the initial phase of the planning process.

Based on the information obtained from the sponsor interview, user interviews, and the review of some existing portfolios, we have defined the current users of the Artist Center web site as coming from different artistic backgrounds: visual artists, writers, and performance artists. The goal of the organization is to provide a venue for these artists to create online portfolios displaying their creative works.

The three personas represented the different areas of artistic concentration and varying levels of technical expertise.

Persona 1 (Jill) is a Visual Artist. She is an intermediate/advanced computer user and uses the internet regularly. Visual Artists consist of photographers, sculptors, painters, graphic artists, and other artistic areas that appeal to the audience’s visual sense.

Persona 2 (Bob), a Writer, is also considered an intermediate/advanced computer user. His skills are not as advanced as Persona 1 but he is familiar with the internet and navigational tools. Writers consist of literary artists including authors, and poets.

Persona 3 (Alfred) is a Performance Artist and a novice computer user. Al does not have email, nor is he comfortable navigating the internet. His technical skill sets are minimal at best and he typically gets frustrated with difficult tasks. Performance artists consist of musicians, spoken word, and actors.

The Artist Center interface requires that its users create a portfolio by establishing a profile and uploading work samples. There are help options available, but the current interface assumes that the user has a working knowledge of how to navigate the internet. As will be detailed later in this report, many of the users had not used the interface before but each was considered either an intermediate/advanced or novice computer user based on the information obtained from the participant screeners.

Novice UsersThese individuals typically have not visited the website but are familiar with the organization. Their computer skill sets are minimal at best. Most are comfortable with email and simple word documents but are novice in the area of formatting and uploading files. These users will not have much knowledge on how to navigate the interface and will probably need to access the help option frequently.

Intermediate/Advanced Users These users are comfortable with the computer and have a working knowledge of how to navigate the internet. They represent our “existing” users and have already created a profile on the Artist Center page. They will complete tasks to update their information. They will also need to access the help option but possibly with less frequency as the novice user.

Recruitment and ScreeningThe sponsor contact person, Hannah Leatherbury, played a major role in identifying and recruiting possible participants. Hannah sent out a mass email to current users of the Artist Center requesting willing individuals to participate in a usability test. Of those that responded there was a pool of approximately 10 potential participants. Based on the sponsor’s personal knowledge, the participants were initially categorized into areas of artistic background and then into groups based on their level of experience with computers and the interface.

The project team then conducted a preliminary interview of each of the possible participants. The purpose of the pre-screening process was to make a preliminary determination of user eligibility. The most effective results are obtained from participants that mirror real-world users.

Based on user research and personas, the project team created a series of questions that would qualify each of the test participants. Some example questions used in the participant screener included:

How many years of experience do you have with computers?

0 – 1 year 1 – 3 years 3 + years

What do you use your computer for?(Check all that apply)

Checking my email and browsing the internet. Playing games

Word processing

Graphic design

Photo editing

Sound recording

Video editing

Figure 2: Excerpt from participant screener questionnaire

Individuals with extensive computer skills and/or that use the internet on a daily basis were deemed ineligible. It was determined that the real-world user’s skill level is novice to intermediate. Participants that exceeded those levels of expertise would not produce usable data for this particular test. As a result of the participant screener, a total of six users were selected for the usability test. A complete copy of the screener questionnaire can be found in Appendix B.

Table one provides details on the test participants, which were are a cross-section of the web tools users. The group is diverse in gender, age, and computing abilities. The walk-through test user is not included.

Table 1: Test Participant Characteristics

Test Participant Characteristics

User 1 User 2 User 3 User 4 User 5 User 6 *User 7

Age Range 29-45 46-Older 29-45 46-Older 46-Older 46-Older 23-28

Gender Male Male Female Male Female Female Female

Artistic Background

Performance / Spoken Word

Performance / Theater

Performance / Vocalist

Visual / Sculptor

Visual / Sculptor

Performance / Spoken Word / Writer

Writer / Musician

Computer Experience Level

Intermediate / Advanced

Novice Intermediate / Advanced

Novice Intermediate / Advanced

Novice Advanced

Hours spent on internet

20+ / week 5 / hours 20+ / week 5 / week 5-10 / week 5 / week 5-10 hours

Primary Use of Internet

Business / Finance

Business / Research

Social Business Research Business Research

Expected frequency of use of artist center

A few times per year

Once a year Very frequently

Occasionally Occasionally Occasionally Occasionally

* Data from user 7 is supplemental because her computing/internet skills exceed that of our user profile.

Methodology: Description of the testing process

Planning Phase

Heuristic Evaluation Prior to conducting the usability test, the project team conducted a heuristic evaluation on the Artist Center interface. A heuristic evaluation is a test in which usability experts judge the usability of an interface against a specific set of heuristics or principles. The heuristics or principles that the usability experts used were taken from Nielsen’s Ten Usability Heuristics. Table 2 describes in detail Nielsen’s heuristics.

Table 2: Nielsen’s Ten Heuristics Source: Carol Barnum, “Usability Testing and Research.” New York: Longman 2002. © 2002 Pearson Education, Inc.Nielsen’s Ten Usability Heuristics

1. Visibility of System Status The system should always keep users informed about what is going on, through appropriate feedback within reasonable time.

2. Match between system and the real world

The system should speak the users' language, with words, phrases and concepts familiar to the user, rather than system-oriented terms. Follow real-world conventions, making information appear in a natural and logical order.

3. User control and freedom Users often choose system functions by mistake and will need a clearly marked "emergency exit" to leave the unwanted state without having to go through an extended dialogue. Support undo and redo.

4. Consistency and standards Users should not have to wonder whether different words, situations, or actions mean the same thing. Follow platform conventions

5. Error prevention Even better than good error messages is a careful design which prevents a problem from occurring in the first place. Either eliminate error-prone conditions or check for them and present users with a confirmation option before they commit to the action.

6. Recognition rather than recall Minimize the user's memory load by making objects, actions, and options visible. The user should not have to remember information from one part of the dialogue to another. Instructions for use of the system should be visible or easily retrievable whenever appropriate.

7. Flexibility and efficiency of use Accelerators -- unseen by the novice user -- may

Nielsen’s Ten Usability Heuristicsoften speed up the interaction for the expert user such that the system can cater to both inexperienced and experienced users. Allow users to tailor frequent actions.

8. Aesthetic and minimalist design Dialogues should not contain information which is irrelevant or rarely needed. Every extra unit of information in a dialogue competes with the relevant units of information and diminishes their relative visibility.

9. Help users recognize, diagnose, and recover from errors

Error messages should be expressed in plain language (no codes), precisely indicate the problem, and constructively suggest a solution.

10. Help and documentation Even though it is better if the system can be used without documentation, it may be necessary to provide help and documentation. Any such information should be easy to search, focused on the user's task, list concrete steps to be carried out, and not be too large.

The purpose of the heuristic evaluation is to discover any potential usability weaknesses that might intimidate the novice or beginner user from creating an artist profile and uploading work samples on the site. Acting as subject matter experts, the four member project team uncovered some areas of concern that the test participants might encounter while interacting with the Artist Center interface. A detailed description of the findings can be found in Appendix A.

A few of the more significant findings are listed below:

“Completed” message missing. When the artist finishes creating the profile, adding samples, etc., there is no “You are finished” signal. It just leaves them in limbo.

All four pages use confusing graphics. Tabs may be confused as links. These graphics may also confuse the user by making him/her think they are the bottom/footer of the page, when they actually need to scroll down further.

Add Publication Screen misses error. “Never Remove” AND a remove date were added without an error message. (Bug)

Text-heavy instructions in online forms tend to blend in with input areas.

FAQ Page lacks search function.

Scrolling issues. On the Artist profile page user has to scroll because input screen is below the screen fold.

Design of QuestionnairesFour questionnaires were created for the test participants to complete at different stages of the usability test. The goal of the questionnaires were to garner some insight to the users perceptions prior to the actual test, during, and then immediately after the test. Comparing the participant’s answers from the different stages of the test revealed how the experience affected their overall view of the tasks and the interface as a whole.

The Participant Screener was summarized in the User Profile section. The purpose of this questionnaire was to qualify potential test participants based on guidelines set by the user personas and the sponsor interview. All of the questionnaires and participant’s responses can be found in the appendices.

The Pre-Test Questionnaire was given prior to the usability test on the actual day of the test. The purpose of this questionnaire was to obtain some insight of the test participant’s perceptions of the current interface and some of their concerns about the test.

The Post-Task Questionnaire was given immediately following each scenario. This was done so that the participant could express themselves while the test experience was still fresh in their memories. Participants were also given the opportunity to give additional comments after each of these questionnaires.

The Post-Test Questionnaire was administered after the completion of the usability test. This questionnaire mirrored the Pre-Test Questionnaire in that the purpose was to obtain some insight of the test participant’s perceptions about the Artist Center interface after completing the usability test.

Description of the ScenariosBased on the results from the Heuristic Evaluation and the sponsor interview, four test scenarios were generated. The project team determined that four detailed scenarios would be sufficient enough to observe any issues that the test participants might encounter. The difficulty of the tasks increased with the advancement of the scenarios.

The goal of the scenarios was to allow the test participant to familiarize themselves with the interface, followed by allowing them to update or create an artist profile. The final two scenarios required the participants to either upload new work samples or edit existing samples. The participant pool consisted of both new and returning users. New users were given the third scenario which involved uploading new samples, while returning users were asked to complete the fourth scenario which required them to edit their work samples, update their events calendar, and list publication in which their work has appeared.

A summary of each of the scenarios is listed below while actual copies of scenarios 1-4 can be found in Appendix D.

Scenario 1: You’ve been invited by the Southern Arts Federation to post your artist profile on their website, “SouthernArtistry.org.” You have clicked the link in an e-mail to the Artist Center on the website.

1. You need to log in to get started. Do this now.Please remember to “Think Out Loud” while performing these tasks.

2. Before you begin creating your profile, you want to familiarize yourself with the homepage and explore its features. Do so now. For time purposes, please do not read the Artist Guidelines. Again, “Think Out Loud.”

3. You find that you need some instructions on creating your profile. Find these now. When you are ready to move on to the next step, creating your profile, raise your hand

Scenario 2: Now That you’ve successfully logged in and looked around, it’s time to create your artist profile. Remember “Think Out Loud.”

1. You would like to create your artist profile now. This involves entering personal and biographical information as well as an artist statement and quote and other information. For time purposes, please limit your inputs to one sentence. Please do this now.

Scenario 3: Now you can begin to place some of your work on the site. Don’t forget to “Think Out Loud!”

1. You want to place several types of work samples on the site. Please do so now using the samples at the following locations:

Writing Sample: address Audio Sample: address

Video: address Picture: address (**signature Image**)

Scenario 4: You have completed creating your profile and displaying samples of your work on the site. Now you would like to replace some of your samples with newer ones and add some more content.

1. You want to replace your writing sample with the one at the address below. Do this now. Address

2. You want to replace an audio sample with the one below and include an image. Do this now. Audio Image

3. You want to let folks know about an upcoming event where your work will be featured. Do so now.

4. You would like to display your work in a different order than it now appears. Please do this now.

5. Your work has previously been featured in “Southern Artist Today” and “Journal of Southeastern Arts” and you want the public to know about it. Do this now.

SchedulingThe usability tests were administered over a three week period beginning on October 26th, 2006. Tests were completed in sets of three with the exception of User 1, which was conducted individually on October 26th. Users 2, 3, and 4 completed the usability test on November 8th. While the remaining users 5, 6, and 7 completed the test on November 18th, 2006.

Testing Environment & EquipmentIn order to simulate a familiar environment, the project team used a relaxed setting, and the user was placed in a room adjacent to the observation team. This allowed the user to be alone during the testing as they would be in their home or a library cubicle.

This setting included a desk, comfortable chair, the computer, and little else. An important piece of equipment was the phone. The phone acted as a primary device for facilitating the usability test, but it also provided a level of comfort to the testers. In a real world situation, they would likely have access to a phone and would be able to call SouthernArtistry.org for assistance. Some of our participants were returning clients so they were used to this process.

The software necessary to facilitate this test is Internet Explorer. The operating system is Windows XP Professional. The connection speed to the web-service is a high speed connection.

The evaluator room is also furnished with the following equipment:

Three wall-mounted video cameras to allow multiple viewing angles Microphone Desk-mounted camera for close viewing

The Southern Arts Usability Test Team viewed the participants via one-way mirror in the observation room and recording test data using the following equipment:

Audio-visual console Audio channel mixer VHS recorder VHS recorder with editing

capabilities Special effects generator

Microphone Speaker with pickup to evaluator

room printer screen video pickup

Figure 2: A view of the Usability Lab

Roles of the TeamThe testing team is made up of four members, plus a sponsor and a faculty advisor. The members of the team will share a number of tasks in order to maximize the contributions of each team member.

Table 3: Team roles and responsibilities Team Member(s) TaskAll project team members Heuristic review of the site

Test plan and final report writing Screening of test participants Narration during the test (excluding logger) Analysis of test output Participation in final presentation of results

Marty Martin Test Administrator/Facilitator

Primary facilitator for the day of the test, administering the pre-test and post-test surveys and seeing to the needs of the test subject.

Travis Melton Video Support

Responsible for operating the cameras and for editing the final video footage based on analysis by all team members.

Lori Rich Help Desk Test Coordinator

Will remain available to the test participant via phone during the test to provide assistance to the test subject, but will also ask probing questions in order to derive more data about the usability issues that have generated the test subject’s question.

This team member will also provide general troubleshooting services and coordination of activities before and during the test.

Edward Simpson Logger

This team member will operate Morae and will log the events of the test, including comments made by the test subject and observations made by each member of the team during the test.

Because of the focused attention required for this activity, the logger will not directly observe the test subject and will therefore not provide narration.

Findings

We will present the results of our test in several different ways. First, we will present our quantitative findings in a short summary. We will show which users completed the scenarios and the time it took for them to finish. Next we will offer a summary of the test logs (Morae Logs) with the errors listed by category in order of severity. Complete copies of the Morae Logs can be found in Appendix H.

In presenting our qualitative findings, we will summarize the user experience by answering the questions posed in our Test Objectives. Then, we will summarize the Post-Task Questionnaires containing the users’ ratings of each scenario and their respective comments. Finally, we will offer a summary of the users’ final impressions of the Artist Center via the Post-Test Questionnaires.

User 7User 7 was considered an outlier user because she had been responsible for updating her organization’s profile for two years and had extensive computer and internet experience. We have included data from her testing session in the Success and Time Performance and the Test Log Summary tables (Quantitative Findings) because we believe that data provides insight into problems that even an experienced user can encounter. Her data was omitted from the Qualitative Data section because we thought it would skew the perception of the tool’s usability. Her responses on the questionnaires have been taken into account and reflected in the Conclusions and Recommendations section of this report.

Quantitative FindingsIn Scenario 1, users had no difficulty logging on to the Artist Center. However, several users did not realize that the instructions for creating or editing their profiles were displayed onscreen and began to search for them elsewhere.

In Scenario 2, most users did not realize that they simply had to scroll down to begin entering their personal information and tried to use the tabs at the bottom of the screen as links. Once they realized where to begin entering/editing information, most users were able to complete this scenario fairly quickly. Some other problems encountered include the following:

Error messages displayed in area of screen currently not visible Error messages don’t clearly identify error Error messages displayed not near actual error Errors on forms clear all fields, not just field in error Entered their personal information in Representative Information form

In Scenarios 3 and 4, four out of six users did not successfully complete the file uploading process. 3 of those four were unaware that they had not uploaded the files even though one of them had previously used the tool to create her profile. Users had trouble with attaching a thumbnail image vs. using the generic icon and several users mistook the writing sample description for the actual writing sample.

Table 4: Success and Time PerformanceSuccess and Time Performance

User 1 User 2 User 3 User 4 User 5 User 6 User 7††

Success(time)

Success(time)

Success(time)

Success(time)

Success(time)

Success(time)

Success(time)

Scenario 1* Yes

(4:33)

Yes

(11:11)

Yes

(2:18)

Yes

(10:26)

Yes

(8:00)

Yes

(8:03)

Yes

(NA)

Scenario 2† Yes

(9:57)

Yes

(11:48)

Yes

(10:24)

Yes

(7:43)

Yes

(10:14)

Yes

(17:08)

Yes

(NA)

Scenario 3 Yes

(12:21)

No

(14:37)

NA Yes

(20:03)

No**

(11:05)

No

(10:08)

NA

Scenario 4 NA NA No

(15:15)

NA NA NA Yes

(NA)

* The time noted is the time it took for the user to realize completion of the scenario. In Scenario 1, each user found the instructions but was unaware or unsure that that they had found them.

† It is important to note that for time purposes, we had users limit the length of their responses during Scenario 2 (profile creation). This resulted in the following:

a. The time spent on this process was limited.b. This prevented them from getting any error messages regarding too many characters in the fields provided. Receiving this error message could have significantly effected the time spent as the error message only tells how many characters are currently in a given field and not how many are allowed in that field or need to be removed.

** User decided not to upload all file types because she felt they didn’t pertain to her - successfully uploaded two out of four files

†† User 7 is considered an outlier user due to her advanced level of computer experience and use of this site. Time to complete scenarios not applicable.

Severity Ratings

High Severe usability issues that hinder or prevent task completion for most users - Top Priority Changes

Medium May hinder or slow task completion for many users - Re-design for Optimum Usability

Low May cause usability issues for some users - Consider for Future Re-design

Table 5: Test Log Summary Southernartistry.org/artistcenter Usability Test – Test Log Summary

Category Scen Sev Description Users Log #

Excessive Scrolling (Minimalist Design)

2 High Couldn’t navigate past instructions on first page – Didn’t understand that scrolling was required to reach the input form

2 13:27.93 4 14:18.78 6 12:09.65

2 High Clicked on “Back to Top” and “Skip Instructions” when filling out profile. Missed the scrolling requirement past horizontal rule

6 20:16.04

Error Handling 2 High Missed error message on phone number entry (Scrolling required to see it)

3 22:25

3 High Missed input error message when she clicked on “SAVE & CONTINUE” (It displayed up in the middle of the screen, so she had to scroll to see it.)

6 26:16.25

3 Med Error message on forms zeros out all fields 1 52:27.71Error Prevention(We may want to combine these with “recognition and recall.)

2 Low Confused by question about providing mailing address vs. email address (Why not both?)

2 35:30

3 High Confuses writing sample description (or comments section – User 7)with actual writing sample

2 52:033 35:59.405 29:56.877 23:36.60

Southernartistry.org/artistcenter Usability Test – Test Log Summary

Category Scen Sev Description Users Log #

3 High Confuses “title” field for “file name” field – Misses “browse for file”

2 52:03.32

6 40:53.252 Med Mistook “Representative’s information for “Artist

Information” 3 18:394 23:03.185 15:16.46

3 Med Skipped “Add Thumbnail Image” 4 42:55.05 29:56.87

3 Med Skipped “Signature Piece” field 3 42:47.766 39:16.04

3 Med Skipped “Type of Work Sample” field 6 44:00.06High Didn’t see “OR” option to use a generic icon 4 45:37.59

5 29:56.876 44:58.06

Flexibility of Use / Form Behavior

3 High Missed that clicking on “back to top” had cleared all her input to the profile form

6 20:37.85

Med Had to go back and fill in all of representative’s information, even though she only had agency name. (Form requires all fields, but doesn’t indicate that.)

6 23:00.25

Help and Documentation 1 Med Difficulty identifying instructions - Paragraph labeled as “instructions” are not task-oriented

1 24:372 13:27.93

4 Low Unable to find how to reorder samples in FAQ 3 41:38.955 Low User wants way to link to “step by step” instructions 1 24:37.12

Southernartistry.org/artistcenter Usability Test – Test Log Summary

Category Scen Sev Description Users Log #

2 Low Misses instructions to put awards in quotes 5 19:09.873 Med Confused by “file size” instructions on the upload work

samples screen5 31:33.48

Low Confused by term “community art” – Is it the same as public art?

7 15:44.40

Impressions of site (overall) n/a “So far I’m frustrated”“Missing something obvious”“This is my usual experience”Gives up

2 22:00.3221:0929:55

It all looks the same 4 16:59.39It isn’t telling me what I should do 5 14:06.26Too much text 5 09:42.67Artist Time Constraints / Too much information 6 18:25.85

Recognition rather than recall 3 Med Duplicate “signature piece” designations 1 49:522 High Entered personal information in representatives

information table2 30:37.93

3 High Didn’t see “step 1”, “step 2” indicators 2 53:083 39:22

3 Med User hovered over “returning user” side of home page, but actually clicks on left side (text links)

4 31:15.39

4 Low Confused by phone number field in events calendar – Is it artist’s phone number or number at event location

7 30:14.60

Visibility of System Status 3 High Misunderstood that files weren’t uploaded – Task attempted but not accomplished. Causes vary: User 6 used “back” button before task completion. User 3 used navigation bar before task completion.

2 53:08.933 3 38:02.393 6 45:38.25

Southernartistry.org/artistcenter Usability Test – Test Log Summary

Category Scen Sev Description Users Log #

1 Med User unsure that scenario1 was complete 1 42:463 Low Confused by message/link to FAQ about submitting work

samples AFTER she’d just submitted her work sample5 35:30.46

Navigation Standards 2 High Mistook tabs for links 2 27:332 4 19:42.592 5 13:25.872 6 19:01.852 Clicked on “returning user” tab thinking it was a link 6 19:01.852 Med Clicked on links inside instructions expecting to access

form – instead accessed FAQ2 27:26.73

2 Low Clicked on headings expecting links 2 21:04.532 Med Difficulty exiting FAQ 2 45:33.731 3 14:564 Med Difficulty navigating to where she could re-order work

samples3 41:01.00

Qualitative Findings

Summary of Qualitative FindingsThe test’s objectives included generating evidence to answer a number of questions about the usability level of the tool. Along with the quantifying test results to answer the objective questions, the team also observed the behavior of the test participants and solicited comments from them during and after the testing process. These qualitative findings are summarized below:

Can the user navigate the screens easily throughout the portfolio-creation process? How often does he/she hesitate or become confused?

All of the test participants experienced some level of difficulty navigating through the site. The highest levels of frustration were seen at two points:

1. The beginning of the process, when the users encountered the need to scroll in order to find the input form. This major problem at the very beginning set a tone of frustration for all the novice users and some of the intermediate/advanced users.

2. During the “upload work samples” process, when they became confused by the tasks. Three users said that they would have simply given up while trying to complete scenario three (uploading work samples).

Which tasks/subtasks cause the most difficulty for the users?

1. Beginning to build the artist profile was difficult for the test participants in that they had trouble getting started. Once they reached the correct screen, they scanned the instructions for links to where they should begin. The need to scroll was not obvious.

2. All the users made the mistake of clicking on the tab graphics and expressed confusion and frustration when they did not work as links.

3. When the users mistyped some information in one of the form fields, they often struggled to identify and correct their error because they couldn’t see the corresponding error message (scrolling was required to see the message), or because no error message was given.

4. Mistakes on the forms also cleared out ALL the fields on the form, instead of the one that caused the error. The users found re-entering all the information to be a tedious task.

5. The less-experienced users often were unaware that entering the descriptions and comments about the samples or entering the file names of the samples did not actually upload the samples.

6. The users sometimes didn’t know that they had not completed the task. Often, they completely overlooked the “step 1, step 2” indicators, or the requirement to enter a thumbnail graphic or a generic icon.

Can the user easily complete tasks not directly related to uploading content, such as creating keywords to make the profile searchable or adding a “remove date” to content?

Most users had no difficulty choosing keywords for their portfolio. The users that did succeed in completing task four also had no difficulty with the “remove date” option.

Filling out the online forms did generate some confusion in that the users were likely to miss the descriptions/instructions for the form. A number of the users mistook the “representative’s information” section of the profile form for the section where their own personal information is added.

Does the FAQ provide the answer quickly and easily?

1. The FAQ is extensive and provides many answers for the users; however the contextual links to the FAQ are not as precise as the users needed. When the user didn’t find his/her answer from the link, he/she did have some difficulty navigating back to the task screen.

2. One user commented that the FAQ doesn’t clearly spell out step-by-step instructions on how to complete a profile, although it provides a lot of background information.

Does the user understand what he/she is doing and why? Or is he/she just following instructions? Does the system provide enough feedback to the user?

1. The majority of the users, even the intermediate/advanced users, became confused or lost at some point during the portfolio-building process. A particularly important result was the users who did not succeed in uploading the files did not realize this was the case.

2. Some of the users completely missed the requirement for a thumbnail image or generic icon and were confused by the need for an image with a writing or audio sample.

How does a new user interact with the site? Does he/she experience frustration or difficulty in completing the four tasks required to build a portfolio?

Frustration and confusion were nearly unanimous responses to the tasks that were tested. New users believed that they were “missing something obvious” and tended to blame themselves. One user stated that “this is my usual experience with things like these.”

New users either called the help desk for assistance or completely failed in completing the tasks.

What are the user’s opinions of the site after building a portfolio versus prior to attempting the tasks?

1. Intermediate/Advanced Users expressed a moderate level of confidence at using the tool again to update their profiles.

2. All the novice-level users with one exception remained intimidated by the process and said that they’d call the help desk for assistance if and when they ever attempt to update their profile in the future.

Does the user believe he/she needs to read through the printed introductory materials prior to building a portfolio?

Because our purpose was to test the usability of the artist center tool on its own merits, we specifically requested that the users omit reading the Artist Guidelines document. It is our contention that with the proper attention to the principles of usability, the users will not need to read a separate document prior to using the tool.

Two users confirmed the truth of that contention by stating that artists – particularly visual artists and performing artists – do not want to take the time to read instructions. Visual artists are more graphics oriented, and performing artists often have a busy schedule of performance dates and don’t have the time to study a document prior to creating a profile.

Post-Task ResponsesEach user was given a questionnaire after each scenario (Post-Task Questionnaire). These questionnaires solicited subjective quantitative data and gave the user an opportunity to offer any comments on each scenario and the web tool as a whole. Each questionnaire contained two to three questions that asked the user to rate different aspects of the scenarios on a five-point scale with 1 being the most negative response, 3 being a neutral response and 5 being the most positive response. The following charts summarize the responses received from these questionnaires. Copies of the completed questionnaires can be found in Appendix E.

Table 6: Scenario 1 Post-Task ResponsesPost-Task Responses - Scenario 1

User 1 User 2 User 3 User 4 User 5 User 6Response (score)

Response (score)

Response (score)

Response (score)

Response (score)

Response (score)

Initial Reaction

It’s OK/Neutral (2.5)

It’s OK (2)

Very Positive (5)

It’s OK (2)

Pretty Good (4)

Very Negative (1)

Comments Wordy, lot’s of windows to open, tabs confusing

Feeling dumb, should be simple,

Long list off Questions (in FAQ), very thorough

Seems very simple and clean, hate to scroll, monotone

Clean, no distractions, need step-by-step instructions

Confusing, want steps laid out clearly, don’t know where to start

Navigating the Site

Average (3)

Very Difficult (1)

Very Easy (5)

Average (3)

Very Easy (5)

Somewhat difficult (2)

Comments Difficult trying to find task

Can’t find the one thing I need to do

Had to remember to go “Home” from FAQ

none Artists may not be proficient w/ computers

Lots of choices, need it simplified

Table 7: Scenario 2 Post-Task ResponsesPost-Task Responses - Scenario 2

User 1 User 2 User 3 User 4 User 5 User 6Response (score)

Response (score)

Response (score)

Response (score)

Response (score)

Response (score)

Overall Reaction

Very Easy (5)

Somewhat Easy (4)

Somewhat Easy (4)

Average/ Somewhat Easy (3.5)

Average (3)

Somewhat Difficult (2)

Comments Very straightforward, generous w/ amount of info you can input

Once I decided what I was supposed to do….

none none Tabs appear as links - scrolling does not seem immediately apparent

Had to think a lot, have to do this for several sites, don’t want to re-think

I Felt: Very confident (5)

Neutral (3)

Fairly Confident (4)

Neutral (3)

Fairly Confident (4)

Fairly Confident (4)

Comments Very practical, if you can’t write about yourself, you’re a lost cause

Felt Totally Lost at first but once he got started it was easy

none none none none

Felt Process Was

Fairly Straightforward (4)

Fairly Straightforward (4)

Very Easy (5)

Fairly Straightforward (4)

Fairly Straightforward (4)

Fairly Straightforward (4)

Comments none Don’t like fitting into boxes

Need to remind people to have pertinent info ready - a “here’s what you’ll need” section”

none Once I got started…….

In navigating the site, it should tell you to scroll

Table 8: Scenario 3 Post-Task ResponsesPost-Task Responses - Scenario 3

User 1 User 2 User 3* User 4 User 5 User 6Response (score)

Response (score)

Response (score)

Response (score)

Response (score)

Response (score)

Uploading Work

Fairly Easy (4)

Very Difficult (1)

NA Fairly Challenging (2)

Fairly Easy (4)

Very Difficult (1)

Comments <glitch due to incorrect file size not fault of site or user>

NA Brown not drawing eye to right spots, blue pops out, need to vary the type

Being familiar with look of site helped, tutorial would help

Takes a lot of time, would have to go outside <site> to have a lot of work done

I Felt: Very Confident (5) Fairly Confident (4)

NA A little Confused (2)

Fairly Confident (4)

Totally Lost (1)

Comments Second nature NA Didn’t seem clear to me

You would have to know which <files> you want to use

None

Will contact Help Desk in Future:

For Some Tasks (3)

For Some Tasks (3)

NA For Some Tasks (3)

For Some Tasks (3)

NA (0)

Comments none NA none none Will turn it over to someone else

*User 3 had previously created her own profile so she skipped Scenario 3 and proceeded to Scenario 4.

Table 9: Scenario 4 Post-Task ResponsesPost-Task Responses - Scenario 4

User 3Response (score)

Will Use Artist Center Again Strongly Agree (5)

Comments none

Overall, found completed tasks: Very Easy (5)

Comments none

Overall, found using website: Very Easy (5)

Comments none

Table 10: Post-Test ResponsesPost-Task Responses - Scenario 3

User 1 User 2 User 3* User 4 User 5 User 6Response (score)

Response (score)

Response (score)

Response (score)

Response (score)

Response (score)

Ease of Use Easy (4)

Fairly Difficult (2)

Very Easy (5)

Somewhat Difficult (2)

Neither easy nor Difficult (3)

Very Difficult (1)

Have you uploaded files before?If No, why not?

Yes No

No need

Yes No

Don’t know

Yes Yes

How often will you update in the future?

Occasionally (3)

Occasionally (3)

Very Often (5)

Occasionally (3)

Occasionally (3)

Occasionally (3)

Comments and Suggestions

Allow literary/spoken word artists to upload writing samples

Simpler instructions, make it clear where to go next

Need a place for reviews of work or guest book of people who visit site

Consider redesigning, color choices, orientation of pages. Need color contrast, replace brown

Make links easier to use, offer a concise tutorial so one can walk along to become familiar

Simplify, make “artist friendly”, allow artist to view actual profile after submitting

Questions left unanswered

none Probably - did not study whole site

none none none Wasn’t sure it represented me correctly

Conclusions and Recommendations

Overall, the users of the artist center tool were pleased with some features of the site and were excited to participate in a project to improve the usability of the tool. Our conclusions are based on the qualitative and quantitative findings gathered by pre-test interviews, observations made during the test, feedback from the participants via questionnaires and post-test interviews, and the test sessions.

Summary of positive findingsIn some cases, the test participants were pleased by error messages that displayed near the error, that provided an accurate description of the error, and that provided clear instructions on how to correct the problem. All the users were able to use the top navigation panel with minimal problems. They also liked that the definition of some terms, for example “public art” were provided next to the associated input area of the forms. Finally, the users liked the streamlined design of the login page.

It is also important to note that in most cases, users who encountered problems and were unable to complete some tasks still retained a positive impression of the Artist Center, particularly User 3. Users generally recognized the need for this usability study and were excited about participating in the discovery of potential improvements.

Bug Fixes1. Repair the “Reorder Work Samples” function.

The users who completed scenario three were unable to reorder the work samples in scenario four. The “Move To Top” key does not work.

2. In the “Publications” section, make the “removal date” and “never remove” mutually exclusive. Users were able to choose both options with no indication of which option would be the default option.

32

Areas of highest usability concern

Table 11: Areas for usability improvements

33

Area of concern Recommended Fix

Excessive scrolling Too much text on the screen Horizontal line breaks at the bottom

of the screen (under some screen resolutions)

Move instructions to a popup window accessed via a link.

Use “NEXT” and “BACK” buttons between each field, instead of requiring scrolling to the field.

Eliminate all scrolling throughout the tool.

Inadequate error prevention Uploading work sample task not

clearly differentiated from describing work samples

Reorder the tasks – Have the user upload the file prior to entering descriptions.

Provide a step-by-step list of tasks to be completed for uploading and describing work samples.

Inadequate error handling Messages that display in area of

screen not currently visible Messages that don’t clearly identify

the error Inconsistent format of messages

(Some in windows, some in red text)

Messages display not near actual error

Highlight actual errors on screen.

Move error message next to error.

Confusing/Inflexible form behaviors Errors on forms clear all fields, not

just the field in error

Highlight form input error without clearing all fields.

Visibility of system status

Upload task completion not clearly marked

Generic icon option not clearly marked

Reorder upload task to attach file before describing it.

Have system default to generic icon

Shorten and clarify the instructions for the thumbnail image.

Recognition rather than recall Form sections not clearly identified Step indicators not emphasized in

Create a separate input form for representative’s information with clear headers.

34

Area of concern Recommended Fixupload work samples process

Feature EnhancementsOur recommended feature enhancements are based on the heuristic evaluation completed by the project team, comments made by the intermediate/advanced users, plus suggestions from one outlier user. The outlier user’s skills exceeded our user profile, and therefore her data does not appear anywhere else in this report. However, she was able to provide an expert-level opinion for the tool.

1. Provide a preview option for the completed profile.2. Broaden the availability of file types for uploading.3. Provide all artist types the ability to upload all sample types.4. Increase the color contrast for the page graphics5. Allow artists to list more than two websites where their work appears.6. Provide a tutorial walk-through for the portfolio-creation process.7. In the Events Calendar, allow artists to enter a range of dates for events that may last

more than one day.

35

Appendices

A. Heuristic ReviewB. Screener QuestionnaireC. Pre-Test QuestionnairesD. Test ScenariosE. Post Task Questionnaires (Completed)F. Post-Test Questionnaires (Completed)G. Facilitator ScriptH. Morae Logs

Appendix A - Heuristic Review

Heuristic Evaluation of Southernartistry.org/artistcenter

An interface for artists to create a web showcase of their work

Completed by the Southern Arts project team:Marty MartinTravis Melton

Lori RichEdward Simpson

September 21, 2006

Table of Contents

Heuristic Evaluation Overview...........................................................................3Introduction.....................................................................................................................................................................................................3Project Team..................................................................................................................................................................................................3Purpose and Methodology..............................................................................................................................................................................3

Heuristic Evaluation Detail Findings.................................................................41. Visibility of system status...................................................................................................................................................................42. Match between system and the real world.........................................................................................................................................53. User control and freedom...................................................................................................................................................................74. Consistency and standards..............................................................................................................................................................105. Error prevention................................................................................................................................................................................116. Recognition rather than recall..........................................................................................................................................................127. Flexibility and efficiency of use.........................................................................................................................................................138. Aesthetic and minimalist design.......................................................................................................................................................149. Help users recognize, diagnose, and recover from errors...............................................................................................................1610. Help and documentation..............................................................................................................................................................17

Heuristic Evaluation Overview http://www.southernartistry.org/artistcenter

Introduction The following heuristic evaluation of the www.southernartistry.org artist center interface was completed September 9-13, 2006 as a preliminary step in the preparation for a full usability test of the interface. The evaluation was completed by the project team after an interview with the project sponsor, Hannah Leatherbury, Program Manager of SouthernArtistry.org. From this sponsor interview, the evaluation team learned that the interface users are primarily novice to intermediate-level computer users who are full-time artists, and therefore are not required to use a computer as a central task in their day to day work lives.

The goal of the users is to publish a web-based showcase of their work on the www.southernartistry.org web site. To accomplish their goal, the Artist Center web pages must carefully guide them through the site building tasks of uploading samples of their work and of building interesting and inviting content to accompany their work samples.

Project TeamSponsor Hannah Leatherbury, Program

Manager, SouthernArtistry.orgAdvisor Dr. Carol Barnum,

Southern Polytechnic State UniversityCoordinator / Writer Lori Rich, IDC Graduate StudentVideo Production / Writer Travis Melton, TCOM Undergraduate

StudentTest Facilitator / Writer Marty Martin, IDC Graduate StudentLogger / Writer Edward Simpson, TCOM

Undergraduate Student

Purpose and MethodologyThe purpose of the heuristic evaluation is to discover any potential usability weaknesses that might intimidate the novice or beginner user from creating an artist showcase on the site. The findings from the evaluation will be used to design appropriate screening questions and scenarios for the usability test.

Based on the sponsor interview, the project team evaluated the site with the novice user in mind. Each of the four segments that a user must complete to post a web showcase were evaluated based on Jakob Neilsen’s 10 Heuristics Adapted for the Web (as quoted in Dr. Barnum’s text, Usability Testing and Research). Issues of concern were ranked on a scale of one to five, with one being the issues of greatest concern.

Severity Ratings: LegendHigh Very High – May cause severe usability issues for all usersMedium High – May cause severe usability issues for some usersLow Moderate – May cause usability issues for some users

Heuristic Evaluation Detail Findings http://www.southernartistry.org/artistcenter

Based on Jakob Neilsen’s 10 Heuristics Adapted for the Web

1. Visibility of system status The system should always keep users informed about what is going on, through appropriate feedback within reasonable time.

Screen elements that pass heuristic: Screen elements with deficiencies:

1- Page titles - All four publication sections have clear titles at the top of each page which allows the user to know exactly what stage of the process they’re in. (Only two use graphic)

2- NavBar formatting

1-URL file name mismatch : Ranking = Low

Same issue is found on almost all the other pages. It’s doubtful that the issue will cause that much confusion from users, since they probably won’t look at the URL once inside the website. This will be one of the minor issues to be tested.

2-“Completed” message missing. Ranking = HighWhen the artist finishes creating the profile, adding samples, etc., there is no “You are finished” signal. It just leaves them in limbo. Adding an indicator that the profile is complete would improve the site.

2. Match between system and the real world The system should speak the users' language, with words, phrases and concepts familiar to the user, rather than system-oriented terms. Follow real-world conventions, making information appear in a natural and logical order. Screen elements that pass heuristic: Screens with deficiencies:

1-Introduction page1-“Publications” page title may confuse less experienced users until they click into it and read about the content to be posted. Ranking = Low

2-Work Sample Page“Encoding” term was used on the Work Sample page. User can reference FAQ on the topic at the beginning of the page. Ranking = Medium

Screen elements that pass heuristic: Screens with deficiencies:

3-Work Samples Page: No indicator on uploading a photograph. Ranking = High

3. User control and freedom Users often choose system functions by mistake and will need a clearly marked "emergency exit" to leave the unwanted state without having to go through an extended dialogue. Support undo and redo. Screen elements that pass heuristic: Screen elements with deficiencies:

1-All pages 1-FAQ Page Navigation Ranking = Medium

2-Display icon page: Ranking = Medium

Screen elements that pass heuristic: Screen elements with deficiencies:

3-All pages use “Cancel” to start over. Ranking = Low

Screen elements that pass heuristic: Screen elements with deficiencies:

4-Tech tips are an unnecessary redirect. Ranking = High

4. Consistency and standards Users should not have to wonder whether different words, situations, or actions mean the same thing. Follow platform conventions.

Screen elements that pass heuristic: Screen elements with deficiencies:

1 - All four pagesSimilar vocabulary and screen design is used throughout the site. Signature piece, publications are commonly used terms.

1 - All four pages use confusing graphic. Ranking = High

These graphics may also confuse the user by making him/her think they are the bottom/footer of the page, when they actually need to scroll down further. When the user moves to the “Biography” page, the tab graphics are at the top and are more easily identified as place markers rather than links between sections.

5. Error prevention Even better than good error messages is a careful design which prevents a problem from occurring in the first place.

Screens that pass heuristic: Screens with deficiencies:

1 - Artist Profile

2 - FAQ’s are linked into most sections to prevent errors and questions.

1 - Add Publication Screen misses error. Ranking = High

6. Recognition rather than recall Make objects, actions, and options visible. The user should not have to remember information from one part of the dialogue to another. Instructions for use of the system should be visible or easily retrievable whenever appropriate. Screens that pass heuristic: Screens with deficiencies:

1 – Artist Profile form 1 - Upload audio screen needs explanation. Ranking = Medium

2 - Small font size on all pages. Ranking = LowA larger font would be less intimidating to novice users.

3 - Navigation to next section missing. Ranking = HighAfter completing each section (profile, attach samples, calendar, etc.), it would be helpful if the user were directed to the next step rather than back to the home page. Perhaps they could be given the option to move on to the next step or to skip the next step if they wanted to come back later. Users who get interrupted and want to go back to where they left off may get confused.

7. Flexibility and efficiency of use Accelerators -- unseen by the novice user -- may often speed up the interaction for the expert user such that the system can

cater to both inexperienced and experienced users. Allow users to tailor frequent actions. Screens that pass heuristic: Screens with deficiencies:

Artist profile pageWorks Sample pageBoth pages offered experienced users the opportunity to skip instructions

Work Sample PageSite may be customized by type of artist, and by doing so. This customization can serve to protect less experienced users from seeing options that might confuse them.

8. Aesthetic and minimalist design Dialogues should not contain information which is irrelevant or rarely needed. Every extra unit of information in a dialogue

competes with the relevant units of information and diminishes their relative visibility. Screen elements that pass heuristic: Screen elements with deficiencies:

All four pages – Although user can “skip instructions”, it may be better to have the new user click on a link to see the instructions, rather than having the experienced user close the instructions each time. This element will be tested. Ranking = High

Screen elements that pass heuristic: Screen elements with deficiencies:

Home Page Sentence could be reworded to eliminate the need for “click here”. Ranking = Low

Logon Page includes too much text. Ranking = Low

9. Help users recognize, diagnose, and recover from errors Error messages should be expressed in plain language (no codes), precisely indicate the problem, and constructively suggest

a solution. Screens that pass heuristic: Screens with deficiencies:

All four pagesError messages are clear and pop-up before the user is able to advance.

1 – Characters Error message may confuse users. Ranking = Low”Too many characters” error box could include a message about the maximum number of characters allowed. It doesn’t explicitly state that the number of characters is too high.

10. Help and documentation Even though it is better if the system can be used without documentation, it may be necessary to provide help and documentation.

Any such information should be easy to search, focused on the user's task, list concrete steps to be carried out, and not be too large.Screens that pass heuristic: Screens with deficiencies:

1- FAQ home page Each page offered a link to the FAQ sectionExtensive documentation provided in FAQ

1 - FAQ Page lacks search function. Ranking = High

Screens that pass heuristic: Screens with deficiencies:

Text-heavy instructions in online forms tend to blend in with input areas. Ranking = High

Key Issues and ConclusionsAreas of focus for user testingThe following areas of concerns were ranked three or higher and will be part of the scenarios used in the usability test.

Heuristic Ranking Screen RecommendationsVisibility of system status

High Most likely would be the publications page

Add a “completed” indicator when all four sections have been completed.

Match between system and the real world

High Work Samples screen

Change the word “encoding” to something more understandable by novice computer users.

High Work Samples screen

Add an option to upload a photograph.

User control and freedom

High FAQ Page Add navigation links to go directly back to where the user was working when he/she consulted the FAQ.

Medium Work Samples page Change “Tech Tips” to pop up windows rather than redirecting the user to the FAQ, where they may have trouble navigating back.

Consistency and standards

High All profile input pages Replace the tabs graphic with something not associated as a link, or make the tabs work as links.

Error prevention High Add Publication screen

Do not allow the user to check the “Never Remove” button and also add a “Remove Date”.

Recognition rather than recall

High All profile input pages Add “Next” buttons to move to the next part of the profile creation.

Aesthetic and minimalist design

High Add Profile screen Change the “Skip Instructions” tool to “See instructions” and remove the instructions from the page until this tool is clicked. Add an icon for the “see instructions” link to make it stand out.

Help and Documentation

High FAQ Page(s) Employ a search function for the help.

High Work Samples page Remove the “audio specifications” text and replace it with a popup or call out.

Overall assessment of the artist center interfaceOur heuristic evaluation of the artist center web interface revealed that there are a number of usability issues that our team can test for with our selected users. Among the most prominent findings were these:

The interface uses a consistent design throughout the profile-creation process.

The interface includes ample documentation for users who are willing to read it.

We found the site to be “text-heavy”. Novice users may be intimidated by the amount that they are required to read while building their profiles.

There are a few graphic elements that may confuse a novice or impatient user.

The pages redirect the user to the FAQ many times. Ideally, the answers the user needs should be intuitive to the page design or available without too much clicking.

Our team looks forward to testing our findings with actual users of the artist center pages.

Appendix B - Screener Questionnaire

Screener Questions

Section 1: Participant Information

Name:

Email: Phone:

Section 2: Computer Experience

Have you ever heard of the Southern Arts Federation? Yes No

How many years of experience do you have with computers?

0 – 1 year 1 – 3 years 3 + years

What do you use your Computer for?(Check all that apply)

Checking my email and browsing the internet.

Playing games

Word processing

Graphic design

Photo editing

Sound recording

Video editing

Do you currently have an online portfolio of any kind? Yes No

Do you belong to any online journal such as LiveJournal, MySpace, or Facebook?

Yes No

Appendix C - Pre-Test Questionnaires

Appendix D - Test ScenariosItems in blue will be defined at time of test.Before beginning, we would like to ask you to “Think Out Loud” while viewing the site and performing any of the required tasks. Tell us what you see, describe what you are doing, walk us through your process. Please note tasks that seem particularly easy to accomplish, as well as those that seem more challenging.

** Please be assured that changes made to your profile during this test will not be permanent. The site administrator will ensure that your current profile and work samples will remain intact.

New User Task List:

Scenario 1: You’ve been invited by the Southern Arts Federation to post your artist profile on their website, “SouthernArtistry.org.” You have clicked the link in an e-mail to the Artist Center on the website.

1. You need to log in to get started. Do this now.Please remember to “Think Out Loud” while performing these tasks.

2. Before you begin creating your profile, you want to familiarize yourself with the homepage and explore its features. Do so now. For time purposes, please do not read the Artist Guidelines. Again, “Think Out Loud.”

3. You find that you need some instructions on creating your profile. Find these now. When you are ready to move on to the next step, creating your profile, raise your hand.

Scenario 2: Now That you’ve successfully logged in and looked around, it’s time to create your artist profile. Remember “Think Out Loud.”

1. You would like to create your artist profile now. This involves entering personal and biographical information as well as an artist statement and quote and other information. For time purposes, please limit your inputs to one sentence. Please do this now.

Scenario 3: Now you can begin to place some of your work on the site. Don’t forget to “Think Out Loud!”

2. You want to place several types of work samples on the site. Please do so now using the samples at the following locations:

Writing Sample: address Audio Sample: address

Video: address Picture: address (**signature Image**)

Scenario 4: You have completed creating your profile and displaying samples of your work on the site. Now you would like to replace some of your samples with newer ones and add some more content.

1. You want to replace your writing sample with the one at the address below. Do this now. Address

2. You want to replace an audio sample with the one below and include an image. Do this now. Audio Image

3. You want to let folks know about an upcoming event where your work will be featured. Do so now.

4. You would like to display your work in a different order than it now appears. Please do this now.

5. Your work has previously been featured in “Southern Artist Today” and “Journal of Southeastern Arts” and you want the public to know about it. Do this now.

Appendix E - Post-Task Questionnaires

Appendix F - Post-Test Questionnaires

Appendix G - Facilitator ScriptFacilitator Script

Thanks for participating in our test. Remember, we are testing the website not you so please be relaxed.

There are all kinds of equipment we’ll be using during the test so I’ll take a moment to familiarize you with it.

Desk, chair Computer, probably similar to one you may have used before Telephone for Help Desk Calls : use just like you would if you were at

home or the office and you needed to call a help desk- give it a try Cameras - 3 on the walls, one on the desk we use these to record

gestures facial expressions. Also we have a view of the same monitor view you have so we can see exactly what you see on your screen. We can also see you through the mirror behind you.

Microphone so we can hear you speaking Is all of this OK with you or do you have any questions about what I’ve

described so far?

Sign consent form

Test Process

First off, I want to assure you that none of the changes you make to your profile during this test will be permanent. The website administrator knows to reject any of the changes made for the purposes of this test.

We really want to get a feel for what it is like for you to use the website so we’ve designed the test with this in mind. Right now I want to give you a description of the test and how it works.

Test Description: You’ll have 4 scenarios (or 3) to complete for the test. Each scenario may have several different tasks associated with it. An example of a task might be logging in to the website.

We need to know what you are thinking during this process so I’m going to ask that you do a few things for us while you move through the scenarios

Read each scenario and task description out loud and then stop reading when you coma to a point where you take action

As you are taking an action, Think Out Loud. Let us know what you think about what you are seeing and doing. Examples “That seemed easy” or “that was difficult” or “not sure where to go here” or “this is a breeze”. Give us your thoughts while you work.

Some of the information you will be asked to enter will allow for some rather lengthy responses. Since we do have limited time, we are asking that you limit your response to these to one or two sentences. Remember, none of this will be saved to your actual profile or portfolio, so biographical accuracy isn’t important here.

When you have completed a scenario, please raise your hand and verbally let us know you are finished and I’ll come back and we’ll talk about what you think of the site up until that point

Once we are finished with all of the scenarios, we’ll do a brief Post-Test Questionnaire and we’ll be good to go.

Do you have any questions about anything at all?

Appendix H - Testing Session Logs (Morae)

User 1 (Pilot User) Morae Log

0:20:22.92,"Marker",,,,,,,,,,,"orientation has been lasting about 15mins. ","Ed Simpson","S","Start task","Remote Viewer",,,"18:48:11 / Oct 26, 2006"0:20:46.71,"Marker",,,,,,,,,,,"Barnum has noted that orientation is taking too long","Ed Simpson","N","Narrator comment","Remote Viewer",,,"18:48:35 / Oct 26, 2006"0:22:50.73,"Marker",,,,,,,,,,,"user has ended the orientation","Ed Simpson","E","End task","Remote Viewer",,,"18:50:39 / Oct 26, 2006"0:23:02.53,"Marker",,,,,,,,,,,"user has logged on to the system","Ed Simpson","S","Start task","Remote Viewer",,,"18:50:51 / Oct 26, 2006"0:23:47.17,"Marker",,,,,,,,,,,"user is reading intro but didn't read the entire thing","Ed Simpson","N","Narrator comment","Remote Viewer",,,"18:51:36 / Oct 26, 2006"0:24:37.12,"Marker",,,,,,,,,,,"user went to FAQ and Help and there's no direction in this section","Ed Simpson","N","Narrator comment","Remote Viewer",,,"18:52:26 / Oct 26, 2006"0:25:29.73,"Marker",,,,,,,,,,,"user is not thinking out loud","Ed Simpson","N","Narrator comment","Remote Viewer",,,"18:53:18 / Oct 26, 2006"0:26:05.32,"Marker",,,,,,,,,,,"user utilized the back button to navigate the FAQ section","Ed Simpson","N","Narrator comment","Remote Viewer",,,"18:53:54 / Oct 26, 2006"0:26:36.92,"Marker",,,,,,,,,,,"user has concerned facial expressions but is not having any problems navigating the site","Ed Simpson","N","Narrator comment","Remote Viewer",,,"18:54:25 / Oct 26, 2006"0:27:35.92,"Marker",,,,,,,,,,,"end of scenario 1","Ed Simpson","E","End task","Remote Viewer",,,"18:55:24 / Oct 26, 2006"0:31:26.32,"Marker",,,,,,,,,,,"start scenario 2","Ed Simpson","S","Start task","Remote Viewer",,,"18:59:15 / Oct 26, 2006"0:31:43.32,"Marker",,,,,,,,,,,"entering information fairly easily ","Ed Simpson","N","Narrator comment","Remote Viewer",,,"18:59:32 / Oct 26, 2006"0:32:28.32,"Marker",,,,,,,,,,,"user is accurately thinking outloud","Ed Simpson","N","Narrator comment","Remote Viewer",,,"19:00:17 / Oct 26, 2006"0:32:44.71,"Marker",,,,,,,,,,,"mentioned that this is easy","Ed Simpson","N","Narrator comment","Remote Viewer",,,"19:00:33 / Oct 26, 2006"0:33:01.51,"Marker",,,,,,,,,,,"very aggressive behavior, moving through the task fairly easily.","Ed Simpson","N","Narrator comment","Remote Viewer",,,"19:00:50 / Oct 26, 2006"0:33:27.32,"Marker",,,,,,,,,,,"body language: leaning into the computer, typing heavily, facial expressions are light.","Ed Simpson","N","Narrator comment","Remote Viewer",,,"19:01:16 / Oct 26, 2006"0:35:06.92,"Marker",,,,,,,,,,,"noted the drop down menu but it isn't actually a drop down menu","Ed Simpson","N","Narrator comment","Remote Viewer",,,"19:02:55 / Oct 26, 2006"0:35:28.92,"Marker",,,,,,,,,,,"very detailed bio and artist statement","Ed Simpson","N","Narrator comment","Remote Viewer",,,"19:03:17 / Oct 26, 2006"0:36:33.71,"Marker",,,,,,,,,,,"body language: interested in topic, bobbing head, tapping table","Ed Simpson","N","Narrator comment","Remote

User 1 (Pilot User) Morae LogViewer",,,"19:04:22 / Oct 26, 2006"0:38:01.92,"Marker",,,,,,,,,,,"noted that there needs to be more selectios for the general services section","Ed Simpson","N","Narrator comment","Remote Viewer",,,"19:05:50 / Oct 26, 2006"0:38:28.51,"Marker",,,,,,,,,,,"noticed that he had five and is searching for the best six choices","Ed Simpson","N","Narrator comment","Remote Viewer",,,"19:06:17 / Oct 26, 2006"0:38:51.92,"Marker",,,,,,,,,,,"user is taking the task very seriously","Ed Simpson","N","Narrator comment","Remote Viewer",,,"19:06:40 / Oct 26, 2006"0:39:40.32,"Marker",,,,,,,,,,,"ask for website twice","Ed Simpson","N","Narrator comment","Remote Viewer",,,"19:07:29 / Oct 26, 2006"0:39:54.92,"Marker",,,,,,,,,,,"clicked on the definition of public arts ","Ed Simpson","N","Narrator comment","Remote Viewer",,,"19:07:43 / Oct 26, 2006"0:40:08.10,"Marker",,,,,,,,,,,"hit the back button to get out of FAQ","Ed Simpson","N","Narrator comment","Remote Viewer",,,"19:07:57 / Oct 26, 2006"0:40:26.92,"Marker",,,,,,,,,,,"body language exudes confidence","Ed Simpson","N","Narrator comment","Remote Viewer",,,"19:08:15 / Oct 26, 2006"0:40:51.12,"Marker",,,,,,,,,,,"give lots of detail even though we mentioned that it wasn't necessary","Ed Simpson","N","Narrator comment","Remote Viewer",,,"19:08:40 / Oct 26, 2006"0:41:22.92,"Marker",,,,,,,,,,,"user has ended scenario 2","Ed Simpson","E","End task","Remote Viewer",,,"19:09:11 / Oct 26, 2006"0:41:39.12,"Marker",,,,,,,,,,,"reading through the steps at the end of the scenario","Ed Simpson","N","Narrator comment","Remote Viewer",,,"19:09:28 / Oct 26, 2006"0:41:51.93,"Marker",,,,,,,,,,,"pretty thorough user","Ed Simpson","N","Narrator comment","Remote Viewer",,,"19:09:40 / Oct 26, 2006"0:42:06.73,"Marker",,,,,,,,,,,"user not sure if he's ended the task","Ed Simpson","N","Narrator comment","Remote Viewer",,,"19:09:55 / Oct 26, 2006"0:42:46.32,"Marker",,,,,,,,,,,"has no sense of the end of the task. there's not notification that says that he's completed the task","Ed Simpson","N","Narrator comment","Remote Viewer",,,"19:10:35 / Oct 26, 2006"0:46:23.85,"Marker",,,,,,,,,,,"start of scenario 3","Ed Simpson","S","Start task","Remote Viewer",,,"19:14:12 / Oct 26, 2006"0:46:33.71,"Marker",,,,,,,,,,,"humming/could mean that he's unsure","Ed Simpson","N","Narrator comment","Remote Viewer",,,"19:14:22 / Oct 26, 2006"0:46:57.53,"Marker",,,,,,,,,,,"he skipped straight to video even though","Ed Simpson","N","Narrator comment","Remote Viewer",,,"19:14:46 / Oct 26, 2006"0:47:18.31,"Marker",,,,,,,,,,,"said audio but didn't click it on the system","Ed Simpson","N","Narrator comment","Remote Viewer",,,"19:15:07 / Oct 26, 2006"0:47:30.71,"Marker",,,,,,,,,,,"user misspoke but is typing movie which is what he clicked","Ed Simpson","N","Narrator comment","Remote Viewer",,,"19:15:19 / Oct 26, 2006"

User 1 (Pilot User) Morae Log0:47:51.92,"Marker",,,,,,,,,,,"tapping when he's unsure","Ed Simpson","N","Narrator comment","Remote Viewer",,,"19:15:40 / Oct 26, 2006"0:48:25.31,"Marker",,,,,,,,,,,"quickly scrolling over the description of the files","Ed Simpson","N","Narrator comment","Remote Viewer",,,"19:16:14 / Oct 26, 2006"0:48:43.92,"Marker",,,,,,,,,,,"has a confused look on his face","Ed Simpson","N","Narrator comment","Remote Viewer",,,"19:16:32 / Oct 26, 2006"0:49:06.53,"Marker",,,,,,,,,,,"found the movie....was excited about finding it","Ed Simpson","N","Narrator comment","Remote Viewer",,,"19:16:55 / Oct 26, 2006"0:49:27.73,"Marker",,,,,,,,,,,"humming to himsef","Ed Simpson","N","Narrator comment","Remote Viewer",,,"19:17:16 / Oct 26, 2006"0:49:34.73,"Marker",,,,,,,,,,,"looking back at the scenario-good reaction","Ed Simpson","N","Narrator comment","Remote Viewer",,,"19:17:23 / Oct 26, 2006"0:49:52.73,"Marker",,,,,,,,,,,"selected the next piece as his signature piece","Ed Simpson","N","Narrator comment","Remote Viewer",,,"19:17:41 / Oct 26, 2006"0:50:19.92,"Marker",,,,,,,,,,,"high energy","Ed Simpson","N","Narrator comment","Remote Viewer",,,"19:18:08 / Oct 26, 2006"0:50:28.92,"Marker",,,,,,,,,,,"nothing is stressing him out. having positive experience. very computer literate","Ed Simpson","N","Narrator comment","Remote Viewer",,,"19:18:17 / Oct 26, 2006"0:51:03.73,"Marker",,,,,,,,,,,"once he got confused after he was told it was too big he had too go back","Ed Simpson","N","Narrator comment","Remote Viewer",,,"19:18:52 / Oct 26, 2006"0:52:27.71,"Marker",,,,,,,,,,,"error recovery","Ed Simpson","N","Narrator comment","Remote Viewer",,,"19:20:16 / Oct 26, 2006"0:52:31.75,"Marker",,,,,,,,,,,"this is highlighted as his signature piece but he picked the video as his signature.","Ed Simpson","N","Narrator comment","Remote Viewer",,,"19:20:20 / Oct 26, 2006"0:53:39.12,"Marker",,,,,,,,,,,"very confident throughout the process","Ed Simpson","N","Narrator comment","Remote Viewer",,,"19:21:28 / Oct 26, 2006"0:54:06.93,"Marker",,,,,,,,,,,"he did not go back and make it his signature image","Ed Simpson","N","Narrator comment","Remote Viewer",,,"19:21:55 / Oct 26, 2006"0:54:20.92,"Marker",,,,,,,,,,,"trying to edit a work sample that is not his task write now","Ed Simpson","N","Narrator comment","Remote Viewer",,,"19:22:09 / Oct 26, 2006"0:55:28.12,"Marker",,,,,,,,,,,"user is showing confusion about adding a writing sample but there's no option for him to do so","Ed Simpson","N","Narrator comment","Remote Viewer",,,"19:23:17 / Oct 26, 2006"0:56:13.73,"Marker",,,,,,,,,,,"demeanor has changed. tapping nervously, hand on the forehand","Ed Simpson","N","Narrator comment","Remote Viewer",,,"19:24:02 / Oct 26, 2006"0:56:35.93,"Marker",,,,,,,,,,,"he's trying to place a writing in another file type","Ed Simpson","N","Narrator comment","Remote Viewer",,,"19:24:24 / Oct 26, 2006"0:56:56.92,"Marker",,,,,,,,,,,"trying to rationalize his actions","Ed Simpson","N","Narrator comment","Remote Viewer",,,"19:24:45 / Oct 26, 2006"

User 1 (Pilot User) Morae Log0:57:07.71,"Marker",,,,,,,,,,,"he's singing which would dictate that he's confident of his actions","Ed Simpson","N","Narrator comment","Remote Viewer",,,"19:24:56 / Oct 26, 2006"0:57:40.32,"Marker",,,,,,,,,,,"got error message. using the phone to call to rectify problem","Ed Simpson","N","Narrator comment","Remote Viewer",,,"19:25:29 / Oct 26, 2006"0:58:43.71,"Marker",,,,,,,,,,,"end of scenario 3","Ed Simpson","E","End task","Remote Viewer",,,"19:26:32 / Oct 26, 2006"0:58:53.12,"Marker",,,,,,,,,,,"problem he had last time he was on the site","Ed Simpson","N","Narrator comment","Remote Viewer",,,"19:26:42 / Oct 26, 2006"0:59:03.71,"Marker",,,,,,,,,,,"1. as a story teller he's mainly verbal but there might be a time that he has a poem or written work. 2. fairly easy, but glitch was an issue 3. fairly confident-confused about the question of signature piece. 4. help desk question is leading. (delete this question) ","Ed Simpson","N","Narrator comment","Remote Viewer",,,"19:26:52 / Oct 26, 2006"1:03:43.92,"Marker",,,,,,,,,,,"comprehensive, usable","Ed Simpson","N","Narrator comment","Remote Viewer",,,"19:31:32 / Oct 26, 2006"

User 2 Morae Log - John Schmedes (Novice-Level)

0:02:40.73,"Marker",,,,,,,,,,,"strart orientation","Ed Simpson","S","Start task","Remote Viewer",,,"15:42:13 / Nov 08, 2006"0:12:11.73,"Marker",,,,,,,,,,,"user starts scenario 1","Ed Simpson","S","Start task","Remote Viewer",,,"15:51:44 / Nov 08, 2006"0:13:27.93,"Marker",,,,,,,,,,,"user is lost trying to find instructions of how to edit profile but they are not on the artist profile page","Ed Simpson","Q","Quote, User comment","Remote Viewer",,,"15:53:00 / Nov 08, 2006"0:19:06.12,"Marker",,,,,,,,,,,"spent a long time on this task","Ed Simpson","Q","Quote, User comment","Remote Viewer",,,"15:58:39 / Nov 08, 2006"0:19:17.93,"Marker",,,,,,,,,,,"using the internet browser to locate instructions","Ed Simpson","Q","Quote, User comment","Remote Viewer",,,"15:58:50 / Nov 08, 2006"0:21:04.53,"Marker",,,,,,,,,,,"trying to click tabs that are not links","Ed Simpson","Q","Quote, User comment","Remote Viewer",,,"16:00:37 / Nov 08, 2006"0:21:22.54,"Marker",,,,,,,,,,,"end of scenario","Ed Simpson","E","End task","Remote Viewer",,,"16:00:55 / Nov 08, 2006"0:21:39.93,"Marker",,,,,,,,,,,"went to FAQ at this time","Ed Simpson","Q","Quote, User comment","Remote Viewer",,,"16:01:12 / Nov 08, 2006"0:22:00.32,"Marker",,,,,,,,,,,"user notes that he is frustrated with the site at this point","Ed Simpson","Q","Quote, User comment","Remote Viewer",,,"16:01:33 / Nov 08, 2006"0:22:25.14,"Marker",,,,,,,,,,,"expected to see something about editing profile per the instructions but was unable to locate that icon","Ed Simpson","Q","Quote, User comment","Remote Viewer",,,"16:01:58 / Nov 08, 2006"0:22:58.32,"Marker",,,,,,,,,,,"after the scenario he finally goes into the FAQ","Ed Simpson","Q","Quote, User comment","Remote Viewer",,,"16:02:31 / Nov 08, 2006"0:23:12.32,"Marker",,,,,,,,,,,"feels like he's missing something that is very obvious","Ed Simpson","Q","Quote, User comment","Remote Viewer",,,"16:02:45 / Nov 08, 2006"0:26:46.14,"Marker",,,,,,,,,,,"start of scenario 2","Ed Simpson","S","Start task","Remote Viewer",,,"16:06:19 / Nov 08, 2006"0:27:26.73,"Marker",,,,,,,,,,,"accessed FAQ but decided against by using back button","Ed Simpson","Q","Quote, User comment","Remote Viewer",,,"16:06:59 / Nov 08, 2006"0:27:54.73,"Marker",,,,,,,,,,,"user contacted help desk. Lori is trying to guide the user by asking what he expects to see on the site. user eventually located the task by scrolling down. ","Ed Simpson","Q","Quote, User comment","Remote Viewer",,,"16:07:27 / Nov 08, 2006"0:30:37.93,"Marker",,,,,,,,,,,"user stays on the phone because he's uncomfortable to work alone","Ed Simpson","Q","Quote, User comment","Remote Viewer",,,"16:10:10 / Nov 08, 2006"0:30:53.70,"Marker",,,,,,,,,,,"lori ended the call against the user's preference","Ed Simpson","Q","Quote, User comment","Remote Viewer",,,"16:10:26 / Nov 08, 2006"0:31:38.32,"Marker",,,,,,,,,,,"notes that this is his usual experience on the site. problems entering his information","Ed Simpson","Q","Quote, User comment","Remote Viewer",,,"16:11:11 / Nov 08, 2006"

User 2 Morae Log - John Schmedes (Novice-Level)0:32:31.12,"Marker",,,,,,,,,,,"'waste of time' user makes comment","Ed Simpson","Q","Quote, User comment","Remote Viewer",,,"16:12:04 / Nov 08, 2006"0:33:00.12,"Marker",,,,,,,,,,,"phone number error message","Ed Simpson","Q","Quote, User comment","Remote Viewer",,,"16:12:33 / Nov 08, 2006"0:33:54.12,"Marker",,,,,,,,,,,"relieved that he's advanced to the next step","Ed Simpson","Q","Quote, User comment","Remote Viewer",,,"16:13:27 / Nov 08, 2006"0:34:16.53,"Marker",,,,,,,,,,,"did not calcuate characters because he knew that it wasn't too many","Ed Simpson","Q","Quote, User comment","Remote Viewer",,,"16:13:49 / Nov 08, 2006"0:34:48.92,"Marker",,,,,,,,,,,"had no problem determining key words even though he's not an experience computer user","Ed Simpson","Q","Quote, User comment","Remote Viewer",,,"16:14:21 / Nov 08, 2006"0:35:18.53,"Marker",,,,,,,,,,,"understanding the language not asscociated with the site itself","Ed Simpson","Q","Quote, User comment","Remote Viewer",,,"16:14:51 / Nov 08, 2006"0:35:38.14,"Marker",,,,,,,,,,,"concerned about definition of public art but didn't look it up","Ed Simpson","Q","Quote, User comment","Remote Viewer",,,"16:15:11 / Nov 08, 2006"0:36:38.73,"Marker",,,,,,,,,,,"he's not tabbing to move forward to next field","Ed Simpson","Q","Quote, User comment","Remote Viewer",,,"16:16:11 / Nov 08, 2006"0:37:30.92,"Marker",,,,,,,,,,,"question 4 is confusing him because of the wording","Ed Simpson","Q","Quote, User comment","Remote Viewer",,,"16:17:03 / Nov 08, 2006"0:38:01.73,"Marker",,,,,,,,,,,"user completed scenario 2","Ed Simpson","Q","Quote, User comment","Remote Viewer",,,"16:17:34 / Nov 08, 2006"0:38:24.92,"Marker",,,,,,,,,,,"end of scenario 2","Ed Simpson","E","End task","Remote Viewer",,,"16:17:57 / Nov 08, 2006"0:39:11.92,"Marker",,,,,,,,,,,"noted that the scenario was easy once he decided how to move to the next step. ","Ed Simpson","Q","Quote, User comment","Remote Viewer",,,"16:18:44 / Nov 08, 2006"0:40:38.73,"Marker",,,,,,,,,,,"Neutral about the task because he started of difficult and then found his stride","Ed Simpson","Q","Quote, User comment","Remote Viewer",,,"16:20:11 / Nov 08, 2006"0:41:08.93,"Marker",,,,,,,,,,,"fairly easy. dislikes the biographical questions. prefers freedom to describe himself.","Ed Simpson","Q","Quote, User comment","Remote Viewer",,,"16:20:41 / Nov 08, 2006"0:43:23.93,"Marker",,,,,,,,,,,"started scenario 3","Ed Simpson","S","Start task","Remote Viewer",,,"16:22:56 / Nov 08, 2006"0:43:35.73,"Marker",,,,,,,,,,,"thinking that his options will be on the right versus what was listed in the left column","Ed Simpson","Q","Quote, User comment","Remote Viewer",,,"16:23:08 / Nov 08, 2006"0:45:15.12,"Marker",,,,,,,,,,,"went to FAQ","Ed Simpson","Q","Quote, User comment","Remote Viewer",,,"16:24:48 / Nov 08, 2006"0:45:19.73,"Marker",,,,,,,,,,,"don't know if he meant to access this link","Ed Simpson","Q","Quote, User comment","Remote Viewer",,,"16:24:52 / Nov 08, 2006"

User 2 Morae Log - John Schmedes (Novice-Level)0:45:33.73,"Marker",,,,,,,,,,,"Stuck in FAQ but he did scroll after issues in the previous scenario.","Ed Simpson","Q","Quote, User comment","Remote Viewer",,,"16:25:06 / Nov 08, 2006"0:46:11.73,"Marker",,,,,,,,,,,"used the address bar to locate the writing sample","Ed Simpson","Q","Quote, User comment","Remote Viewer",,,"16:25:44 / Nov 08, 2006"0:46:42.25,"Marker",,,,,,,,,,,"left the site completely","Ed Simpson","Q","Quote, User comment","Remote Viewer",,,"16:26:15 / Nov 08, 2006"0:46:53.54,"Marker",,,,,,,,,,,"totally off the site but eventually used the back button to get back to the site","Ed Simpson","Q","Quote, User comment","Remote Viewer",,,"16:26:26 / Nov 08, 2006"0:47:33.59,"Marker",,,,,,,,,,,"hovered over the work sample link but did not click it","Ed Simpson","Q","Quote, User comment","Remote Viewer",,,"16:27:06 / Nov 08, 2006"0:47:49.32,"Marker",,,,,,,,,,,"finally found the link","Ed Simpson","Q","Quote, User comment","Remote Viewer",,,"16:27:22 / Nov 08, 2006"0:47:56.00,"Marker",,,,,,,,,,,"repeated his previous error by confusing the 'text' link (FAQ) as a 'task' link","Ed Simpson","Q","Quote, User comment","Remote Viewer",,,"16:27:29 / Nov 08, 2006"0:48:46.32,"Marker",,,,,,,,,,,"called help desk. ","Ed Simpson","Q","Quote, User comment","Remote Viewer",,,"16:28:19 / Nov 08, 2006"0:49:30.73,"Marker",,,,,,,,,,,"trying to select unclickable tabs","Ed Simpson","Q","Quote, User comment","Remote Viewer",,,"16:29:03 / Nov 08, 2006"0:49:40.53,"Marker",,,,,,,,,,,"user found the next task without the help of the helpdesk","Ed Simpson","Q","Quote, User comment","Remote Viewer",,,"16:29:13 / Nov 08, 2006"0:49:57.73,"Marker",,,,,,,,,,,"has issue with you scrolling down","Ed Simpson","Q","Quote, User comment","Remote Viewer",,,"16:29:30 / Nov 08, 2006"0:50:15.73,"Marker",,,,,,,,,,,"user found signature pce question","Ed Simpson","Q","Quote, User comment","Remote Viewer",,,"16:29:48 / Nov 08, 2006"0:50:27.34,"Marker",,,,,,,,,,,"user advances through the step with ease once he found his way","Ed Simpson","Q","Quote, User comment","Remote Viewer",,,"16:30:00 / Nov 08, 2006"0:52:03.32,"Marker",,,,,,,,,,,"he thinks that the descriptive text is the writing sample. he moves on to the next task without uploading the file.","Ed Simpson","Q","Quote, User comment","Remote Viewer",,,"16:31:36 / Nov 08, 2006"0:52:41.53,"Marker",,,,,,,,,,,"doesn't understand that he's supposed to access the files from the desktop","Ed Simpson","Q","Quote, User comment","Remote Viewer",,,"16:32:14 / Nov 08, 2006"0:53:08.93,"Marker",,,,,,,,,,,"not finishing the tasks because he's not seeing the 'STEP one ' 'STEP two' indicators","Ed Simpson","Q","Quote, User comment","Remote Viewer",,,"16:32:41 / Nov 08, 2006"0:54:18.92,"Marker",,,,,,,,,,,"error message about copywrite. which said that it needed to be in correct format","Ed Simpson","Q","Quote, User comment","Remote Viewer",,,"16:33:51 / Nov 08, 2006"0:55:10.32,"Marker",,,,,,,,,,,"not following the scenario tasks at this point.","Ed Simpson","Q","Quote, User comment","Remote Viewer",,,"16:34:43 /

User 2 Morae Log - John Schmedes (Novice-Level)Nov 08, 2006"0:56:06.93,"Marker",,,,,,,,,,,"he questions why a comment is needed if he doesn't want to put on there","Ed Simpson","Q","Quote, User comment","Remote Viewer",,,"16:35:39 / Nov 08, 2006"0:56:47.32,"Marker",,,,,,,,,,,"not seeing the step two indicator","Ed Simpson","Q","Quote, User comment","Remote Viewer",,,"16:36:20 / Nov 08, 2006"0:57:50.73,"Marker",,,,,,,,,,,"end of scenario 3","Ed Simpson","E","End task","Remote Viewer",,,"16:37:23 / Nov 08, 2006"1:00:09.54,"Marker",,,,,,,,,,,"doesn't like the constraints of not being able to use his own words. gets easily confused on websites. confusing and frustrating were the descriptive words. ","Ed Simpson","Q","Quote, User comment","Remote Viewer",,,"16:39:42 / Nov 08, 2006"

User 3 Morae Log - Aretha Lockhart (Intermediate-Level Computer Skills)

0:13:18.79,"Marker",,,,,,,,,,,"started scenario 1 13.26 mins behind the begining of recording","Ed Simpson","S","Start task","Remote Viewer",,,"19:39:41 / Nov 08, 2006"0:14:09.03,"Marker",,,,,,,,,,,"user in FAQ and clicked getting started and thought that it was an actual task link","Ed Simpson","Q","Quote, User comment","Remote Viewer",,,"19:40:32 / Nov 08, 2006"0:14:56.67,"Marker",,,,,,,,,,,"user stuck in the FAQ right from the start","Ed Simpson","Q","Quote, User comment","Remote Viewer",,,"19:41:19 / Nov 08, 2006"0:15:17.78,"Marker",,,,,,,,,,,"user said that she would try to find her way back after getting stuck in FAQ","Ed Simpson","Q","Quote, User comment","Remote Viewer",,,"19:41:40 / Nov 08, 2006"0:15:40.39,"Marker",,,,,,,,,,,"user finished scenario 1 and skipped ahead","Ed Simpson","X","Error, Unexpected action","Remote Viewer",,,"19:42:03 / Nov 08, 2006"0:16:18.98,"Marker",,,,,,,,,,,"end of scenario 2","Ed Simpson","E","End task","Remote Viewer",,,"19:42:41 / Nov 08, 2006"0:16:44.18,"Marker",,,,,,,,,,,"very positive outlook about the site. very thorough questions at the begining","Ed Simpson","Q","Quote, User comment","Remote Viewer",,,"19:43:07 / Nov 08, 2006"0:17:30.39,"Marker",,,,,,,,,,,"we might have been wrong about her being lost in the FAQ section","Ed Simpson","Q","Quote, User comment","Remote Viewer",,,"19:43:53 / Nov 08, 2006"0:17:47.98,"Marker",,,,,,,,,,,"noted that the FAQ was very good and noted that she had to remember how to get out of that section","Ed Simpson","Q","Quote, User comment","Remote Viewer",,,"19:44:10 / Nov 08, 2006"0:18:25.79,"Marker",,,,,,,,,,,"start of scenario 2","Ed Simpson","S","Start task","Remote Viewer",,,"19:44:48 / Nov 08, 2006"0:18:39.20,"Marker",,,,,,,,,,,"she might be unsure if she knows that she's entering in her representatives info rather than her own","Ed Simpson","Q","Quote, User comment","Remote Viewer",,,"19:45:02 / Nov 08, 2006"0:20:34.39,"Marker",,,,,,,,,,,"user got phone # error and she realized her errors ","Ed Simpson","X","Error, Unexpected action","Remote Viewer",,,"19:46:57 / Nov 08, 2006"0:22:25.59,"Marker",,,,,,,,,,,"system will not take an 1-800 number must be in normal format. Error message is non specific and user has no idea which number is the cause of the error","Ed Simpson","X","Error, Unexpected action","Remote Viewer",,,"19:48:48 / Nov 08, 2006"0:24:24.79,"Marker",,,,,,,,,,,"note: users have not used the tip link at all","Ed Simpson","Q","Quote, User comment","Remote Viewer",,,"19:50:47 / Nov 08, 2006"0:25:40.59,"Marker",,,,,,,,,,,"user is very diligent about following the directions on the screen. Calculates content","Ed Simpson","Q","Quote, User comment","Remote Viewer",,,"19:52:03 / Nov 08, 2006"0:26:50.20,"Marker",,,,,,,,,,,"the wording in the text box my seem like a command rather than an option. 'Calculate Characters'","Ed Simpson","Q","Quote, User comment","Remote Viewer",,,"19:53:13 / Nov 08, 2006"0:28:10.39,"Marker",,,,,,,,,,,"end of scenario 2","Ed Simpson","E","End task","Remote Viewer",,,"19:54:33 / Nov 08, 2006"

0:29:44.59,"Marker",,,,,,,,,,,"end of scn2","Ed Simpson","E","End task","Remote Viewer",,,"19:56:07 / Nov 08, 2006"0:31:15.00,"Marker",,,,,,,,,,,"somewhat easy (scenario 2) felt that she was going through the scenarios too quickly. Fairly confident for the second question. very easy for third question. Suggested that a listing of items that might be asked for during the artist profile. 'Here's what you'll need' section","Ed Simpson","Q","Quote, User comment","Remote Viewer",,,"19:57:38 / Nov 08, 2006"0:35:12.20,"Marker",,,,,,,,,,,"she seemed very knowledgeable about uploading docs, vid, and music","Ed Simpson","Q","Quote, User comment","Remote Viewer",,,"20:01:35 / Nov 08, 2006"0:35:31.98,"Marker",,,,,,,,,,,"start of scenario 4","Ed Simpson","S","Start task","Remote Viewer",,,"20:01:54 / Nov 08, 2006"0:35:40.79,"Marker",,,,,,,,,,,"did not use the next button. used the top level navigation ","Ed Simpson","Q","Quote, User comment","Remote Viewer",,,"20:02:03 / Nov 08, 2006"0:35:59.40,"Marker",,,,,,,,,,,"confusing the title with file name. might just be wording of the scenario","Ed Simpson","Q","Quote, User comment","Remote Viewer",,,"20:02:22 / Nov 08, 2006"0:36:59.79,"Marker",,,,,,,,,,,"she thinks she uploaded the files but she didn't do it","Ed Simpson","X","Error, Unexpected action","Remote Viewer",,,"20:03:22 / Nov 08, 2006"0:38:02.39,"Marker",,,,,,,,,,,"all of the users are skipping step 2. she did it in all three steps","Ed Simpson","Q","Quote, User comment","Remote Viewer",,,"20:04:25 / Nov 08, 2006"0:39:22.39,"Marker",,,,,,,,,,,"users are not seeing STEP TWO on the site","Ed Simpson","Q","Quote, User comment","Remote Viewer",,,"20:05:45 / Nov 08, 2006"0:41:01.00,"Marker",,,,,,,,,,,"went back to the home page and then asked how she is supposed to change the list","Ed Simpson","Q","Quote, User comment","Remote Viewer",,,"20:07:24 / Nov 08, 2006"0:41:38.95,"Marker",,,,,,,,,,,"went into FAQ but under the wrong topic","Ed Simpson","Q","Quote, User comment","Remote Viewer",,,"20:08:01 / Nov 08, 2006"0:41:51.48,"Marker",,,,,,,,,,,"realized that she will not find the answer in this spot","Ed Simpson","Q","Quote, User comment","Remote Viewer",,,"20:08:14 / Nov 08, 2006"0:42:31.43,"Marker",,,,,,,,,,,"user called help desk ","Ed Simpson","H","User needs help","Remote Viewer",,,"20:08:54 / Nov 08, 2006"0:42:47.76,"Marker",,,,,,,,,,,"went into FAQ at the signature piece","Ed Simpson","Q","Quote, User comment","Remote Viewer",,,"20:09:10 / Nov 08, 2006"0:43:04.98,"Marker",,,,,,,,,,,"nothing on the system that says reorder your work","Ed Simpson","Q","Quote, User comment","Remote Viewer",,,"20:09:27 / Nov 08, 2006"0:46:18.20,"Marker",,,,,,,,,,,"nothing is happening when she selected the action buttons","Ed Simpson","Q","Quote, User comment","Remote Viewer",,,"20:12:41 / Nov 08, 2006"0:47:10.39,"Marker",,,,,,,,,,,"user remembered the publication section and immediately navigated to this tab ","Ed Simpson","Q","Quote, User comment","Remote Viewer",,,"20:13:33 / Nov 08, 2006"0:48:00.39,"Marker",,,,,,,,,,,"under publication detail the system will allow you enter a remove pub. date and check never remove this publication without an error message","Ed Simpson","Q","Quote, User comment","Remote Viewer",,,"20:14:23 / Nov 08, 2006"

0:50:46.39,"Marker",,,,,,,,,,,"end of scenario 4","Ed Simpson","E","End task","Remote Viewer",,,"20:17:09 / Nov 08, 2006"0:50:53.40,"Marker",,,,,,,,,,,"1. agree 2. very easy 3. very easy ","Ed Simpson","Q","Quote, User comment","Remote Viewer",,,"20:17:16 / Nov 08, 2006"0:52:17.59,"Marker",,,,,,,,,,,"user is very PRO-SITE. Work sample section, she felt it was too much at one time. ","Ed Simpson","Q","Quote, User comment","Remote Viewer",,,"20:18:40 / Nov 08, 2006"0:53:59.98,"Marker",,,,,,,,,,,"felt frustrated about uploading files","Ed Simpson","Q","Quote, User comment","Remote Viewer",,,"20:20:22 / Nov 08, 2006"0:59:55.59,"Marker",,,,,,,,,,,"empowering, time-saving were the cards that she chose.","Ed Simpson","Q","Quote, User comment","Remote Viewer",,,"20:26:18 / Nov 08, 2006"1:01:29.57,"Marker",,,,,,,,,,,"again comments were very pro-site. commented on the number of files that represented her type of work","Ed Simpson","Q","Quote, User comment","Remote Viewer",,,"20:27:52 / Nov 08, 2006"

User 4 Morae Logs - Mario Petririna (Novice-Level)

0:06:35.79,"Marker",,,,,,,,,,,"start of scenario 1","Ed Simpson","S","Start task","Remote Viewer",,,"20:42:10 / Nov 08, 2006"0:13:05.98,"Marker",,,,,,,,,,,"user used the tab button during log on","Ed Simpson","Q","Quote, User comment","Remote Viewer",,,"20:48:40 / Nov 08, 2006"0:14:18.78,"Marker",,,,,,,,,,,"user is somewhat confused about directions. facial expression shows his confusion. called the help desk for clarification","Ed Simpson","Q","Quote, User comment","Remote Viewer",,,"20:49:53 / Nov 08, 2006"0:16:51.39,"Marker",,,,,,,,,,,"end of scenario 1","Ed Simpson","E","End task","Remote Viewer",,,"20:52:26 / Nov 08, 2006"0:16:59.39,"Marker",,,,,,,,,,,"1. has a 'just ok' view about the site. user noted that he's a visual person and needs simplicity, hates to scroll, and the interface is not geared towards an artist. Size of the print and the design. blames his insecurity for his views. 2. Navigating site is average. ","Ed Simpson","Q","Quote, User comment","Remote Viewer",,,"20:52:34 / Nov 08, 2006"0:19:34.18,"Marker",,,,,,,,,,,"start of scenario 2","Ed Simpson","S","Start task","Remote Viewer",,,"20:55:09 / Nov 08, 2006"0:19:42.59,"Marker",,,,,,,,,,,"user clicked on the tab but it's not a link","Ed Simpson","Q","Quote, User comment","Remote Viewer",,,"20:55:17 / Nov 08, 2006"0:20:35.59,"Marker",,,,,,,,,,,"continues to try to click tabs but not able to access it from that point","Ed Simpson","Q","Quote, User comment","Remote Viewer",,,"20:56:10 / Nov 08, 2006"0:21:59.59,"Marker",,,,,,,,,,,"called help desk about finding the artist profile, user located the information by scrolling down","Ed Simpson","H","User needs help","Remote Viewer",,,"20:57:34 / Nov 08, 2006"0:23:03.18,"Marker",,,,,,,,,,,"user is trying to advance with out completing the form and gets an error message","Ed Simpson","Q","Quote, User comment","Remote Viewer",,,"20:58:38 / Nov 08, 2006"0:23:48.78,"Marker",,,,,,,,,,,"uses the enter key in place of the next button","Ed Simpson","Q","Quote, User comment","Remote Viewer",,,"20:59:23 / Nov 08, 2006"0:24:05.20,"Marker",,,,,,,,,,,"is not phased about the calculate buttons like previous users","Ed Simpson","Q","Quote, User comment","Remote Viewer",,,"20:59:40 / Nov 08, 2006"0:25:00.79,"Marker",,,,,,,,,,,"has six adjectives selected and did not read the limit from the directions","Ed Simpson","Q","Quote, User comment","Remote Viewer",,,"21:00:35 / Nov 08, 2006"0:25:35.20,"Marker",,,,,,,,,,,"got an error message and it took him back to the location of the error. He noted that he liked that it told him that he made an error","Ed Simpson","Q","Quote, User comment","Remote Viewer",,,"21:01:10 / Nov 08, 2006"0:27:17.59,"Marker",,,,,,,,,,,"end of scenario 2","Ed Simpson","E","End task","Remote Viewer",,,"21:02:52 / Nov 08, 2006"0:27:24.79,"Marker",,,,,,,,,,,"1.average to somewhat easy. issues with the visual appearance. less pro site. 2. neutral, his insecurities 3. fairly straight","Ed Simpson","Q","Quote, User comment","Remote Viewer",,,"21:02:59 / Nov 08, 2006"0:30:55.98,"Marker",,,,,,,,,,,"start scenario 3","Ed Simpson","S","Start task","Remote Viewer",,,"21:06:30 / Nov 08, 2006"

User 4 Morae Logs - Mario Petririna (Novice-Level)0:31:15.39,"Marker",,,,,,,,,,,"noted that he has no idea what to do but knew to navigate back to the homepage.","Ed Simpson","Q","Quote, User comment","Remote Viewer",,,"21:06:50 / Nov 08, 2006"0:31:58.00,"Marker",,,,,,,,,,,"didn' t know where to go and immediately called the helpdesk for help and then located the add work sample option","Ed Simpson","H","User needs help","Remote Viewer",,,"21:07:33 / Nov 08, 2006"0:33:15.20,"Marker",,,,,,,,,,,"recognized question about sig. piece","Ed Simpson","Q","Quote, User comment","Remote Viewer",,,"21:08:50 / Nov 08, 2006"0:33:59.20,"Marker",,,,,,,,,,,"gave a frustrating sigh about this type of work","Ed Simpson","Q","Quote, User comment","Remote Viewer",,,"21:09:34 / Nov 08, 2006"0:34:15.20,"Marker",,,,,,,,,,,"went to FAQ because he wanted an example of the description and used the back button successfully","Ed Simpson","Q","Quote, User comment","Remote Viewer",,,"21:09:50 / Nov 08, 2006"0:34:37.20,"Marker",,,,,,,,,,,"he used the back button to look for directions. has a confused facial expression","Ed Simpson","Q","Quote, User comment","Remote Viewer",,,"21:10:12 / Nov 08, 2006"0:36:25.39,"Marker",,,,,,,,,,,"he successfully uploaded a pic to the system and now he's going back to upload more images","Ed Simpson","Q","Quote, User comment","Remote Viewer",,,"21:12:00 / Nov 08, 2006"0:38:45.79,"Marker",,,,,,,,,,,"user seems a lilttle more confident about the task but did not realize that it is under thumbnail image.","Ed Simpson","Q","Quote, User comment","Remote Viewer",,,"21:14:20 / Nov 08, 2006"0:40:43.20,"Marker",,,,,,,,,,,"noticed the sig. piece the question","Ed Simpson","Q","Quote, User comment","Remote Viewer",,,"21:16:18 / Nov 08, 2006"0:42:55.00,"Marker",,,,,,,,,,,"skipped the thumbnail and successfully uploads the writing sample","Ed Simpson","Q","Quote, User comment","Remote Viewer",,,"21:18:30 / Nov 08, 2006"0:44:15.39,"Marker",,,,,,,,,,,"recognized that the error message deleted his original writing sample","Ed Simpson","Q","Quote, User comment","Remote Viewer",,,"21:19:50 / Nov 08, 2006"0:45:37.59,"Marker",,,,,,,,,,,"user called to question error message and does not read the 'OR' stops at the browse button","Ed Simpson","H","User needs help","Remote Viewer",,,"21:21:12 / Nov 08, 2006"0:47:06.59,"Marker",,,,,,,,,,,"user is stumped by the thumbnail image","Ed Simpson","Q","Quote, User comment","Remote Viewer",,,"21:22:41 / Nov 08, 2006"0:49:33.20,"Marker",,,,,,,,,,,"user recognizes the color scheme of the interface and that this could be overlooked by the user. the 'OR' blends in with the other text and gets overlooked.","Ed Simpson","Q","Quote, User comment","Remote Viewer",,,"21:25:08 / Nov 08, 2006"0:50:59.82,"Marker",,,,,,,,,,,"end of scn. 3","Ed Simpson","E","End task","Remote Viewer",,,"21:26:34 / Nov 08, 2006"0:51:06.40,"Marker",,,,,,,,,,,"1. fairly challenging. The brown coloring blended with the title. suggested varying type. 2. a little confused. just didn't seem clear to him 3.","Ed Simpson","Q","Quote, User comment","Remote Viewer",,,"21:26:41 / Nov 08, 2006"0:52:21.00,"Marker",,,,,,,,,,,"3. for some tasks will contact the help desk ","Ed Simpson","Q","Quote, User comment","Remote Viewer",,,"21:27:56 / Nov 08, 2006"

User 5 Morae Log - Marci (Advanced Computer Skills)

0:01:34.87,"Marker",,,,,,,,,,,"Start scenario 1","Edward ","S","Start task","Remote Viewer",,,"09:15:26 / Nov 18, 2006"0:02:28.87,"Marker",,,,,,,,,,,"user is not using tab button. indication of computer experience","Edward ","S","Start task","Remote Viewer",,,"09:16:20 / Nov 18, 2006"0:02:58.48,"Marker",,,,,,,,,,,"user thinks the links are obvious, clean, no distractions. ","Edward ","A","User action","Remote Viewer",,,"09:16:50 / Nov 18, 2006"0:03:33.07,"Marker",,,,,,,,,,,"user maximizes guidelines. she commented that she shouldn't read the these but she's well into the guidelines.","Edward ","A","User action","Remote Viewer",,,"09:17:25 / Nov 18, 2006"0:04:27.06,"Marker",,,,,,,,,,,"user says that she's not a fan of adobe for visual reasons. ","Edward ","A","User action","Remote Viewer",,,"09:18:19 / Nov 18, 2006"0:04:59.46,"Marker",,,,,,,,,,,"user is still in the guideline section","Edward ","A","User action","Remote Viewer",,,"09:18:51 / Nov 18, 2006"0:05:11.45,"Marker",,,,,,,,,,,"closed window to go back to the home page","Edward ","A","User action","Remote Viewer",,,"09:19:03 / Nov 18, 2006"0:05:49.87,"Marker",,,,,,,,,,,"user did not access faq section (possibly because she didn't notice the link)","Edward ","A","User action","Remote Viewer",,,"09:19:41 / Nov 18, 2006"0:06:24.87,"Marker",,,,,,,,,,,"user is now in the FAQ to research how to upload photo","Edward ","A","User action","Remote Viewer",,,"09:20:16 / Nov 18, 2006"0:06:43.73,"Marker",,,,,,,,,,,"accessed the back to question list link","Edward ","A","User action","Remote Viewer",,,"09:20:35 / Nov 18, 2006"0:07:14.67,"Marker",,,,,,,,,,,"user is more computer savy than normal","Edward ","A","User action","Remote Viewer",,,"09:21:06 / Nov 18, 2006"0:07:47.07,"Marker",,,,,,,,,,,"user noticed that there are easier ways to upload text than what was clarified in the FAQ","Edward ","A","User action","Remote Viewer",,,"09:21:39 / Nov 18, 2006"0:08:22.07,"Marker",,,,,,,,,,,"user used the back button to get back to the hompage","Edward ","A","User action","Remote Viewer",,,"09:22:14 / Nov 18, 2006"0:08:37.46,"Marker",,,,,,,,,,,"user attempted to click the STEP 1 Contact TAB","Edward ","A","User action","Remote Viewer",,,"09:22:29 / Nov 18, 2006"0:08:56.87,"Marker",,,,,,,,,,,"has not yet scrolled down to access the rest of the page","Edward ","A","User action","Remote Viewer",,,"09:22:48 / Nov 18, 2006"0:09:34.06,"Marker",,,,,,,,,,,"end of scenario 1","Edward ","E","End task","Remote Viewer",,,"09:23:26 / Nov 18, 2006"0:09:42.67,"Marker",,,,,,,,,,,"user comments that artist do not typically read a lot of text. (visuals would be more affective)","Edward ","Q","Quote, User comment","Remote Viewer",,,"09:23:34 / Nov 18, 2006"0:10:40.26,"Marker",,,,,,,,,,,"post task 1.pretty good Comments: clean site, but step orientation could be less text heavy. 2. very easy ","Edward ","Q","Quote, User comment","Remote Viewer",,,"09:24:32 / Nov 18, 2006"

User 5 Morae Log - Marci (Advanced Computer Skills)0:12:34.46,"Marker",,,,,,,,,,,"start scenario 2","Edward ","S","Start task","Remote Viewer",,,"09:26:26 / Nov 18, 2006"0:12:43.87,"Marker",,,,,,,,,,,"user is still to access the tab. notice that its not an active link","Edward ","A","User action","Remote Viewer",,,"09:26:35 / Nov 18, 2006"0:13:25.87,"Marker",,,,,,,,,,,"does not yet realize that the tabs are not links.","Edward ","A","User action","Remote Viewer",,,"09:27:17 / Nov 18, 2006"0:14:06.26,"Marker",,,,,,,,,,,"skipped the artist profile and went to work samples but then used the back button","Edward ","A","User action","Remote Viewer",,,"09:27:58 / Nov 18, 2006"0:14:33.07,"Marker",,,,,,,,,,,"wasn't clear that she had to scroll down to access the profile","Edward ","A","User action","Remote Viewer",,,"09:28:25 / Nov 18, 2006"0:14:49.06,"Marker",,,,,,,,,,,"comments that the tabs are misleading","Edward ","A","User action","Remote Viewer",,,"09:28:41 / Nov 18, 2006"0:15:16.46,"Marker",,,,,,,,,,,"entering her own information in the Representative info section. misleading to the user","Edward ","A","User action","Remote Viewer",,,"09:29:08 / Nov 18, 2006"0:15:45.06,"Marker",,,,,,,,,,,"user got an error message for Password information","Edward ","A","User action","Remote Viewer",,,"09:29:37 / Nov 18, 2006"0:17:24.67,"Marker",,,,,,,,,,,"located the error and made changes","Edward ","A","User action","Remote Viewer",,,"09:31:16 / Nov 18, 2006"0:17:33.67,"Marker",,,,,,,,,,,"skip the calculate characters button because of the statement she made earlier about there being an easier way to do it","Edward ","A","User action","Remote Viewer",,,"09:31:25 / Nov 18, 2006"0:18:57.87,"Marker",,,,,,,,,,,"skipped keywords","Edward ","A","User action","Remote Viewer",,,"09:32:49 / Nov 18, 2006"0:19:09.87,"Marker",,,,,,,,,,,"no user has read the directions about putting awards in quotes. directions are getting overlooked by the users","Edward ","A","User action","Remote Viewer",,,"09:33:01 / Nov 18, 2006"0:20:53.87,"Marker",,,,,,,,,,,"not using the definitions for each of the the sections after the awards box","Edward ","A","User action","Remote Viewer",,,"09:34:45 / Nov 18, 2006"0:21:16.67,"Marker",,,,,,,,,,,"user completed updating her profile. ","Edward ","A","User action","Remote Viewer",,,"09:35:08 / Nov 18, 2006"0:22:48.67,"Marker",,,,,,,,,,,"end of scenario 2","Edward ","E","End task","Remote Viewer",,,"09:36:40 / Nov 18, 2006"0:23:00.86,"Marker",,,,,,,,,,,"post task questions 1. average Comments: Tabs are not links 2. confident 3. fairly easy ","Edward ","Q","Quote, User comment","Remote Viewer",,,"09:36:52 / Nov 18, 2006"0:24:56.46,"Marker",,,,,,,,,,,"'Once I got started....' a comment used by many users. They're finding that getting the process moving is difficult but once they've moved passed the obstacles it becomes easy","Edward ","Q","Quote, User comment","Remote Viewer",,,"09:38:48 / Nov 18, 2006"0:25:51.50,"Marker",,,,,,,,,,,"start scenario 3","Edward ","S","Start task","Remote Viewer",,,"09:39:43 / Nov 18, 2006"0:26:00.87,"Marker",,,,,,,,,,,"questions where to attatch work samples","Edward ","A","User action","Remote Viewer",,,"09:39:52 / Nov 18, 2006"0:26:43.87,"Marker",,,,,,,,,,,"she's criticizing the task page and not the site","Edward ","A","User action","Remote Viewer",,,"09:40:35 / Nov 18, 2006"

User 5 Morae Log - Marci (Advanced Computer Skills)0:27:06.06,"Marker",,,,,,,,,,,"user read the directions for photo credit. 'not yourself'","Edward ","A","User action","Remote Viewer",,,"09:40:58 / Nov 18, 2006"0:28:10.07,"Marker",,,,,,,,,,,"no one is going in order per the task list","Edward ","A","User action","Remote Viewer",,,"09:42:02 / Nov 18, 2006"0:28:33.06,"Marker",,,,,,,,,,,"bad navigation. thought the FAQ was the next step instead of uploading another file","Edward ","A","User action","Remote Viewer",,,"09:42:25 / Nov 18, 2006"0:29:27.07,"Marker",,,,,,,,,,,"hit the 'next' link and uploaded a writing sample","Edward ","A","User action","Remote Viewer",,,"09:43:19 / Nov 18, 2006"0:29:56.87,"Marker",,,,,,,,,,,"uploading the writing sample in the thumbnail section","Edward ","A","User action","Remote Viewer",,,"09:43:48 / Nov 18, 2006"0:30:39.67,"Marker",,,,,,,,,,,"she noticed the error afterwards","Edward ","A","User action","Remote Viewer",,,"09:44:31 / Nov 18, 2006"0:30:55.26,"Marker",,,,,,,,,,,"she's not reading the entire set of directions","Edward ","A","User action","Remote Viewer",,,"09:44:47 / Nov 18, 2006"0:31:33.48,"Marker",,,,,,,,,,,"finds the thumbnail option frustrating because of unfamiliarity with file sizes.","Edward ","A","User action","Remote Viewer",,,"09:45:25 / Nov 18, 2006"0:34:20.26,"Marker",,,,,,,,,,,"after user was 'coached' by the helpdesk she was able to correct her thumbnail mistake","Edward ","A","User action","Remote Viewer",,,"09:48:12 / Nov 18, 2006"0:35:04.45,"Marker",,,,,,,,,,,"previewed the writing sample after upload is complete","Edward ","A","User action","Remote Viewer",,,"09:48:56 / Nov 18, 2006"0:35:30.46,"Marker",,,,,,,,,,,"unsure of what to do after you complete the upload. questions the numerous steps to complete the entire process. to much for an artist","Edward ","A","User action","Remote Viewer",,,"09:49:22 / Nov 18, 2006"0:36:55.76,"Marker",,,,,,,,,,,"end of scenario 3","Edward ","E","End task","Remote Viewer",,,"09:50:47 / Nov 18, 2006"0:37:06.06,"Marker",,,,,,,,,,,"user thought that what she was typing was the actual writing sample","Edward ","Q","Quote, User comment","Remote Viewer",,,"09:50:58 / Nov 18, 2006"0:39:38.28,"Marker",,,,,,,,,,,"commented that there is too much text on the site that the user gets lost in the data. Makes it difficult to find the exact task the user wants to accomplish","Edward ","Q","Quote, User comment","Remote Viewer",,,"09:53:30 / Nov 18, 2006"0:44:12.67,"Marker",,,,,,,,,,,"during the test users facial expressions were very stoic. no noticeable expressions","Edward ","Q","Quote, User comment","Remote Viewer",,,"09:58:04 / Nov 18, 2006"0:45:06.07,"Marker",,,,,,,,,,,"usable, busy. Comments: site is usable once you understand the site and how to navigate it. Busy because there's too much at one time.","Edward ","Q","Quote, User comment","Remote Viewer",,,"09:58:58 / Nov 18, 2006"

User 6 Morae Logs - Cathy Kaemmerlen (Novice Level Skills)0:06:45.45,"Marker",,,,,,,,,,,"start scenario 1","Edward ","S","Start task","Remote Viewer",,,"10:19:48 / Nov 18, 2006"0:08:26.84,"Marker",,,,,,,,,,,"did not realize that she was on the homepage","Edward ","A","User action","Remote Viewer",,,"10:21:29 / Nov 18, 2006"0:11:08.85,"Marker",,,,,,,,,,,"reviewing the other links artist profile, work samples, etc.","Edward ","A","User action","Remote Viewer",,,"10:24:11 / Nov 18, 2006"0:11:34.45,"Marker",,,,,,,,,,,"instead of exploring the homepage she's exploring the entire site.","Edward ","A","User action","Remote Viewer",,,"10:24:37 / Nov 18, 2006"0:12:09.65,"Marker",,,,,,,,,,,"clicking on title tabs assuming that they are link","Edward ","A","User action","Remote Viewer",,,"10:25:12 / Nov 18, 2006"0:13:09.25,"Marker",,,,,,,,,,,"feels that she is lost. has completed task 1 but doesn't realize it. ","Edward ","A","User action","Remote Viewer",,,"10:26:12 / Nov 18, 2006"0:14:11.45,"Marker",,,,,,,,,,,"user is frowning and seems frustrated","Edward ","A","User action","Remote Viewer",,,"10:27:14 / Nov 18, 2006"0:14:23.06,"Marker",,,,,,,,,,,"end of task","Edward ","A","User action","Remote Viewer",,,"10:27:26 / Nov 18, 2006"0:15:12.26,"Marker",,,,,,,,,,,"end of scenario 1","Edward ","E","End task","Remote Viewer",,,"10:28:15 / Nov 18, 2006"0:15:19.65,"Marker",,,,,,,,,,,"1. between negative, and ok Comment: confusing. artist have limited time and patience. Wants something simple and easy to use. doesn't know how to start 2. somewhat difficult","Edward ","Q","Quote, User comment","Remote Viewer",,,"10:28:22 / Nov 18, 2006"0:17:56.45,"Marker",,,,,,,,,,,"start of scenario 2","Edward ","S","Start task","Remote Viewer",,,"10:30:59 / Nov 18, 2006"0:18:25.85,"Marker",,,,,,,,,,,"does not look forward to updating profile information because of the difficulty level and time that it takes to update","Edward ","Q","Quote, User comment","Remote Viewer",,,"10:31:28 / Nov 18, 2006"0:19:01.85,"Marker",,,,,,,,,,,"attempts to access the tabs that are not actually links. the tabs have scroll over info.","Edward ","A","User action","Remote Viewer",,,"10:32:04 / Nov 18, 2006"0:20:16.04,"Marker",,,,,,,,,,,"accessed the skip instructions link","Edward ","A","User action","Remote Viewer",,,"10:33:19 / Nov 18, 2006"0:20:26.65,"Marker",,,,,,,,,,,"only user to use the skip option","Edward ","A","User action","Remote Viewer",,,"10:33:29 / Nov 18, 2006"0:20:37.85,"Marker",,,,,,,,,,,"clicked on back to top but then hit skip instructions again.","Edward ","A","User action","Remote Viewer",,,"10:33:40 / Nov 18, 2006"0:22:01.84,"Marker",,,,,,,,,,,"does not know to scroll","Edward ","A","User action","Remote Viewer",,,"10:35:04 / Nov 18, 2006"0:22:10.85,"Marker",,,,,,,,,,,"same issue as other users. does not see any 'next' or 'continue'","Edward ","A","User action","Remote Viewer",,,"10:35:13 / Nov 18, 2006"0:22:31.65,"Marker",,,,,,,,,,,"confused facial expressions","Edward ","A","User action","Remote Viewer",,,"10:35:34 / Nov 18, 2006"0:22:40.85,"Marker",,,,,,,,,,,"continue to hit back to top and skip instructions","Edward ","A","User action","Remote Viewer",,,"10:35:43 / Nov 18, 2006"

User 6 Morae Logs - Cathy Kaemmerlen (Novice Level Skills)0:23:00.25,"Marker",,,,,,,,,,,"noticed that every time that she hits back to the top it clears her information","Edward ","A","User action","Remote Viewer",,,"10:36:03 / Nov 18, 2006"0:23:54.32,"Marker",,,,,,,,,,,"'where is the save and continue button'","Edward ","A","User action","Remote Viewer",,,"10:36:57 / Nov 18, 2006"0:24:11.65,"Marker",,,,,,,,,,,"actually doesn't realize that her infor is being deleted every time she hits the back button","Edward ","A","User action","Remote Viewer",,,"10:37:14 / Nov 18, 2006"0:25:35.04,"Marker",,,,,,,,,,,"user scrolled down after help desk call","Edward ","A","User action","Remote Viewer",,,"10:38:38 / Nov 18, 2006"0:25:58.45,"Marker",,,,,,,,,,,"realized that the representative info is not her information","Edward ","A","User action","Remote Viewer",,,"10:39:01 / Nov 18, 2006"0:26:16.25,"Marker",,,,,,,,,,,"error message does not pop up but is in red at the top of the screen","Edward ","A","User action","Remote Viewer",,,"10:39:19 / Nov 18, 2006"0:27:10.65,"Marker",,,,,,,,,,,"user does not realize that an error was made","Edward ","A","User action","Remote Viewer",,,"10:40:13 / Nov 18, 2006"0:27:24.25,"Marker",,,,,,,,,,,"profile will not allow her to proceed","Edward ","A","User action","Remote Viewer",,,"10:40:27 / Nov 18, 2006"0:27:55.65,"Marker",,,,,,,,,,,"calls the help desk","Edward ","A","User action","Remote Viewer",,,"10:40:58 / Nov 18, 2006"0:28:03.65,"Marker",,,,,,,,,,,"located the error message: rep address and phone #. The directions does not say that this information is required but if you start the page you must complete it in its entirety","Edward ","A","User action","Remote Viewer",,,"10:41:06 / Nov 18, 2006"0:29:59.04,"Marker",,,,,,,,,,,"bio section she calculated characters","Edward ","A","User action","Remote Viewer",,,"10:43:02 / Nov 18, 2006"0:30:11.85,"Marker",,,,,,,,,,,"did not calculate afterwards because she realizes its relevance","Edward ","A","User action","Remote Viewer",,,"10:43:14 / Nov 18, 2006"0:30:46.85,"Marker",,,,,,,,,,,"accessed FAQ in the keywords section and hit the back button to exit ","Edward ","A","User action","Remote Viewer",,,"10:43:49 / Nov 18, 2006"0:31:13.26,"Marker",,,,,,,,,,,"user scrolls after hovering over back to top option","Edward ","A","User action","Remote Viewer",,,"10:44:16 / Nov 18, 2006"0:32:27.06,"Marker",,,,,,,,,,,"facial expressions are not noticeable now that she knows how to advance","Edward ","A","User action","Remote Viewer",,,"10:45:30 / Nov 18, 2006"0:33:01.45,"Marker",,,,,,,,,,,"comments that the site only allow you to list two websites even though she has more","Edward ","A","User action","Remote Viewer",,,"10:46:04 / Nov 18, 2006"0:33:20.81,"Marker",,,,,,,,,,,"completed the artist profile section","Edward ","A","User action","Remote Viewer",,,"10:46:23 / Nov 18, 2006"0:35:12.06,"Marker",,,,,,,,,,,"end of scenario 2","Edward ","E","End task","Remote Viewer",,,"10:48:15 / Nov 18, 2006"0:35:50.04,"Marker",,,,,,,,,,,"comments that this is painful","Edward ","Q","Quote, User comment","Remote Viewer",,,"10:48:53 / Nov 18, 2006"0:36:00.46,"Marker",,,,,,,,,,,"comments like previous user that 'getting started is frustrating'","Edward ","Q","Quote, User comment","Remote Viewer",,,"10:49:03 / Nov 18, 2006"0:36:23.26,"Marker",,,,,,,,,,,"task questions 1. somewhat difficult Comments: you have to think alot. 2. fairly confident 3.","Edward ","Q","Quote,

User 6 Morae Logs - Cathy Kaemmerlen (Novice Level Skills)User comment","Remote Viewer",,,"10:49:26 / Nov 18, 2006"0:37:50.45,"Marker",,,,,,,,,,,"3. fairly straightforward","Edward ","Q","Quote, User comment","Remote Viewer",,,"10:50:53 / Nov 18, 2006"0:38:00.25,"Marker",,,,,,,,,,,"'scroll down' should be an option and the error message should be more visablle","Edward ","Q","Quote, User comment","Remote Viewer",,,"10:51:03 / Nov 18, 2006"0:39:04.28,"Marker",,,,,,,,,,,"scenario 3","Edward ","S","Start task","Remote Viewer",,,"10:52:07 / Nov 18, 2006"0:39:16.04,"Marker",,,,,,,,,,,"passed over the signature piece question","Edward ","A","User action","Remote Viewer",,,"10:52:19 / Nov 18, 2006"0:40:53.25,"Marker",,,,,,,,,,,"thinks the title is the file name and location","Edward ","A","User action","Remote Viewer",,,"10:53:56 / Nov 18, 2006"0:41:26.65,"Marker",,,,,,,,,,,"does not know that her image is on the desktop","Edward ","A","User action","Remote Viewer",,,"10:54:29 / Nov 18, 2006"0:41:46.06,"Marker",,,,,,,,,,,"went all the way back home when she got confused. Zeroed out all of her information","Edward ","A","User action","Remote Viewer",,,"10:54:49 / Nov 18, 2006"0:42:08.85,"Marker",,,,,,,,,,,"when she does not know what to do she hits the back button. does not realize that she's not uploading her information","Edward ","A","User action","Remote Viewer",,,"10:55:11 / Nov 18, 2006"0:43:07.45,"Marker",,,,,,,,,,,"'does not know how to attatch thumbnail image' she skips over this option","Edward ","A","User action","Remote Viewer",,,"10:56:10 / Nov 18, 2006"0:44:00.06,"Marker",,,,,,,,,,,"User is not reading the title instrucitions. ","Edward ","A","User action","Remote Viewer",,,"10:57:03 / Nov 18, 2006"0:44:58.06,"Marker",,,,,,,,,,,"did not see the 'Or' generic icon","Edward ","A","User action","Remote Viewer",,,"10:58:01 / Nov 18, 2006"0:45:38.25,"Marker",,,,,,,,,,,"user uses the back button becuase she now realize that she has uploaded any information","Edward ","A","User action","Remote Viewer",,,"10:58:41 / Nov 18, 2006"0:46:46.65,"Marker",,,,,,,,,,,"user got an error message because she has not attatched image","Edward ","A","User action","Remote Viewer",,,"10:59:49 / Nov 18, 2006"0:47:08.46,"Marker",,,,,,,,,,,"no wording that says 'ATTATCH'","Edward ","A","User action","Remote Viewer",,,"11:00:11 / Nov 18, 2006"0:48:20.85,"Marker",,,,,,,,,,,"user is extremely confused and frustrated at this point","Edward ","A","User action","Remote Viewer",,,"11:01:23 / Nov 18, 2006"0:49:12.04,"Marker",,,,,,,,,,,"end of scenario 3","Edward ","E","End task","Remote Viewer",,,"11:02:15 / Nov 18, 2006"0:49:21.04,"Marker",,,,,,,,,,,"task questions #3 ","Edward ","Q","Quote, User comment","Remote Viewer",,,"11:02:24 / Nov 18, 2006"0:53:55.85,"Marker",,,,,,,,,,,"frustrating and time consuming. wasn't clear where to go, what to do. time consuming because it takes too much time trying to figure out those things. Felt 'dumb' felt that she HAS to be knowlegeable about computers to use the site","Edward ","Q","Quote, User comment","Remote Viewer",,,"11:06:58 / Nov 18, 2006"