project 1. heuristic evaluation and cognitive walkthrough ... · this report is an analysis of...

PROJECT 1.Heuristic Evaluation and Cognitive Walkthrough

of Goroo.com

Cherese Cooper, Tatiana Iegorova, Andrew WasowiczHCI 460 Usability Evaluations

Spring 2013

1

Executive Summary

In this report we analyzed heuristic evaluations used to develop recommendation to improve thefunctionality of Goroo.com. We based evaluations on the heuristic principles developed by Jacob Nielsen.Each evaluator focused on several functions and features of Goroo.com. We gauged the extent to whichthose functions and features adhered to Nielson’s heuristic principles.

Goroo.com is a travel planning site which provides information on the public transportation systems of theChicago land area. We assume that the user’s immediate goal for using the site is assumed is to retrievetravel information and the user’s extended goal is to travel from one location to another within the Chicagoland area. The evaluators applied ten heuristic criteria while exploring how Goroo.com aided users inacheiving their immediate and extended goals. The ten heuristic principles are:

1. Visibility of system status2. Match between system and real world3. User control and freedom4. Consistency and standards5. Error prevention6. Recognition rather than recall7. Flexibility and efficiency of use8. Aesthetic and minimalist design9. Help with errors10. Provide help (if needed)

After analyzing the evaluations, we have established ten core issues that affect the usability of Goroo.com.These issues are provided in the following report and are coupled with a listing of elements, from the site,which adhered appropriately to the heuristic guidelines. Recommendations are developed to offer solutionsthat will fit with the overall feel and intent of Goroo.com.

After analyzing the evaluations, we have established ten core usability issues:

1. No Status shows when loading different travel methods2. Account creation confirmation looks like an error alert3. Does not show current login status4. Location information requires a specific address5. Provides inaccurate walking/ bicycling/driving directions6. Cancel button appears grayed out when editing trip details7. Using browser back/forward navigation causes problems for results page8. Inconsistencies between query buttons and user constraints9. Sign out option is only accessible inside the user’s account10. There is no cancel button for update account

A report is provided on two cognitive walkthroughs. Evaluators explored cognitive walkthroughs for 1)querying Goroo.com for directions from one loacation to another, and 2) editing the query after results havebeen provided. These Cognitive walkthroughs explore potential user knowledge and how the system worksto allow the user to utilize his or her knowledge to achieve their immediate goals.

After discussing our findings in this report, we compiled a list of the top priority recommendations. The

2

following recommendations should be implemented as soon as possible:1. Make search results persistent to ensure that users can navigate between the pages using browser

controls without losing previous states.

2. Walking/Bicycling/driving instructions need to reflect the actual path the user will need to take.

3. Account set up should display the email notification in an overly window instead of a javascriptalert.

Introduction

This report is an analysis of three heuristic evaluations of Goroo.com. The follow describes the evaluatedsite, and target users. The report presents several problematic aspects of Goroo.com. Additionaldescriptions of features which adhere to Nielsen's heuristics are provided for comparison and to illustrateaspects of the site’s design which were well thought out and which can inform potential solutions.

Product DescriptionGoroo.com is an online site which provides users with travel plans for public transportation in theChicagoland area. The site provides users with travel instructions that include options for buses, trains,driving, walking, and bicycling. The site also provides users with alerts of conditions affecting bus and trainroutes. Users can customize these alerts and receive notifications via email or text message for theirprefered train or bus lines. Goroo.com provides information in English, Polish, and Spanish.

Target PopulationThere are three populations targeted by the developers at Goroo.com.

1. Chicago area residents who rely on public transportation infrequently or those using publictransportation lines with which they are unfamiliar. These users would need the site to queryinformation on efficient ways to navigate the public transportation systems from one location toanother. Included in this population are Spanish and/or Polish speaking residents who may havedifficulty or limited skill speaking English.

2. Out of town tourists. This population is likely to be unfamiliar with the Chicago area and may be aptto query Goroo.com for travel plans to specific attractions in and around Chicago.

3. Users who are familiar with Chicago’s public transportation system or who use specifictransportation lines regularly. The user can set up notification accounts that will push travel alertsto their email accounts or phones, updating them of conditions that are affecting their preferred busand train lines

3

Heuristics & Severity

Heuristic used in the evaluationsNumber Heuristic Description

1 Visibility of system status2 Match between system and the real world3 User control and freedom4 Consistency and standards5 Error prevention6 Recognition rather than recall7 Flexibility and efficiency8 Aesthetic and minimalist design9 Help with errors10 Provide help (if needed)

Severity ratingsRating Description

0 No usability problem1 Cosmetic problem fix only if time allows2 Minor usability problem low priority3 Major usability problem high priority4 Usability catastrophe imperative

Findings

Summary of ProblemsAfter the analysis of the heuristic evaluations, evaluators composed ten usability issues which requireattention. These problems have been rated by their severity. Violated Heuristic principles have been listedfor each problem and a rational has been included to explain why and how each applicable heuristic wasviolated. Similar features which adhere to Nielsen's heuristics have been listed in the full details to illustratehow potential solutions are already present in the design. Explicit recommendations are provided in the fulldetails as guidelines of how each problem could be improved.

4

# Problem Heuristics Severity General Heuristic

1 No Status shows when loadingdifferent travel methods

1 1 Keep the user informed of what thesystem is doing

2 Account creation confirmation lookslike an error alert

1 4 5 2 Suggests an error status, javascriptsalert boxed commonly used for errormessages, may cause the user to err

3 Does not show current login status 1 6 1 Log in status is not visible, Increasescognitive load by forcing user toremember login status

4 Location information requires aspecific address

2 5 7 4 Public transportation is usuallynavigated by general locations, Causeserrors if the user lacks full information,User must conform to rigid interaction

5 Provides inaccurate walking/bicycling/driving directions

2 4 Directions do not match real world paths,these directions are superfluous unlessmade functional

6 Cancel button appears grayed outwhen editing trip details

1 3 3 User is lead to believe that they cannotcancel Edit Trip

7 Using browser back/forwardnavigation causes problems forresults page

3 7 4 Punishes the user for using interface inan unexpected way, User cannot easilyback navigate without causing errors

8 Inconsistencies between querybuttons and user constraints

4 3 different button conventions are used indifferent areas of the site

9 Sign out option is only accessibleinside the user’s account

4 7 4 Does not follow common conventions,user can sign out from only one location,slows down ability to carry out goal

10 There is no cancel button in the editaccount section

3 4 The user is not allowed to cancel the editaccount option.

Full Details of Problem Areas

1. No loading status when changing between travel methods

# Problem Heuristics Severity Details

1 No Status shows when loadingdifferent travel methods

1 1 Keep the user informed of what thesystem is doing

ProblemAfter the user has queried the system, results are displayed providing the user with directions to the desired

5

destination. From this state the user can select different travel options. When switching between theseoptions the site does not provide feedback that the system is loading new information. When the system isfunctioning this causes no problems, but if there is a slow connection or if the system is having problemsloading the information, then the user is not shown adequate feedback that they have performed anappropriate action and must wait for the system to update to be certain.

Rational1) Visibility of system status: Keeping the user informed of changes in system status helps the userunderstand what is happening and what effect their actions have. Without this feedback the user maymistake slow to update information as a failed interaction on the users part. The user may exacerbate aproblem by repeating the action or attempting other unnecessary actions.

Examples of appropriate adherence to heuristics Fields are highlight when in focus Navigation tabs are highlighted when the user is on that particular page Loading animation plays when system retrieves train or bus schedules

RecommendationsThe site should use the same loading animation used for transit

schedules when the user is selecting between travel options.

2. Notification of “confirmation email sent” looks like an error message


2 Account creation confirmation lookslike an error alert

1 4 5 2 Suggests an error status, javascriptsalert boxed commonly used for errormessages, may cause the user to err

ProblemWhen the user has signed up to create an account the system displays a javascript alert informing the userthat a confirmation email has been sent. This is meant to inform the user that they have successfullycreated an account and to instruct the user of the next action they should take to complete the signupprocess. Because the notification looks like an error alert the user is likely to assume that a mistake wasmade. The user may attempt to sign up again only to find another error message telling them that theyalready have an account.

6

Rational1) Visibility of system status: Notifying the user that a confirmation email has been sent informs the userthat the signup process has been successful thus far. By using a javascript alert the system is suggestingthat the task was unsuccessful and that the system is in error status.2) Consistency and standards: The system uses javascript alerts to inform the user of errors in severalother places. The email confirmation notification is the only nonerror javascript alert found. it breaks fromconsistency within the site3) Error prevention: The email confirmation alert may cause users to make mistakes by suggesting thatsomething went wrong in the signup process, prompting the user to take unnecessary actions.

Examples of appropriate adherence to heuristics Icons are consistent with industry standards Icons are used consistently throughout the site Overlay panels appear in areas where the user moves out of the normal task flow but still moving

forward

RecommendationsThe site should display the notification in an overly window as is done during edit trip. Focus of the

confirmation should be prominent, and include colors and icons that would clearly communicate that theuser has successfully completed a signup (e.g. check mark, green colors). And also notify that an emailwas sent to the provided email address.

3. Does not show current login status


3 Does not show current login status 1 6 1 Log in status is not visible, Increasescognitive load by forcing user toremember login status

ProblemGoroo.com allows the user to set up a personal account. Accessing the account requires that the user loginto the site. The site does not dynamically display whether or not the user is logged in. The navigation optionfor the user’s account displays “My Account” both when the user is logged in and logged out.

Rational1) Visibility of system status: The system should display the current status. By displaying “My Account.”With no other indication of logged in status, the system is creating a false user expectation. The user willexpect to be able to access their account immediately and that they are already logged in.2) Recognition rather than recall: The user must remember whether they are logged in. If the user cannotremember the user will need to check his or her account.

7

Examples of appropriate adherence to heuristics Icons are consistent and reduce cognitive load on the user Language status is effectively displays in the upper right corner Navigation tabs are highlighted when the user is on that particular page

RecommendationsThe “My Account” button should dynamically change values from “Sign in” when not logged in, to

“My Account” when the user is logged in. Additionally, there could be a quick link ‘Sign Out” when a user islogged in. That will allow for oneclick sign out, and it would also be a visual cue that the user is currentlysigned in and has an option to sign out.

4. Queries require specific addresses


4 Location information requires aspecific address

2 6 7 4 Public transportation is usuallynavigated by more general location, usermust look up address if unknown, User ismust conform to rigid interaction

ProblemThe site is intended to help users plan their method of transportation. Queries require that users havespecific location information. Users (especially target population 2) may find this difficult if they do not haveexact address.

Rational1) Match between system and real world: Bus and train travel is not an address to address service. Stopsand stations are at set locations and often identified by location descriptors which are more general thanaddresses (Pulaski Station, Rush and Division). By requiring a specific address the system fails to matchreal world concepts of public transportation2) Recognition rather than recall: If a user does not know the actual addresses of locations, then the userwill need to look up the addresses from another resource or rely on selecting nearby attractions. Users willneed to look up the proximity of the attractions (those recognized by the system) to see which matchestheir need if they are unfamiliar with the area. If a user is planning to travel to an attraction then the systemmay work fine but this assumes that the user’s intended attraction is recognized by the system.

8

3) Flexibility and efficiency of use: The rigid requirement for location information prevents the user fromdeveloping a report with the system. The user is limited in their options of interaction methods. Additionallyif the user must look elsewhere to verify and address then the system has lost efficiency of use.

Examples of appropriate adherence to heuristics Order of information input is logical and matches a “from/to” conceptual model of travel Users can input an address or he/she can select from their recent locations for efficiency Users can customize their trip method of travel

RecommendationsAllow users to query by more general terms such as intersections and provide multiple stops from

which they can choose their destination. Provide search results and additional suggested attractionsdepicted on the map. Allow to pick a location from the map and retrieve directions to/from it.

5. Provides inaccurate walking/bicycling/driving directions


5 Provides inaccurate walking/bicycling/driving directions

2 8 4 Directions do not match real world paths,these directions are superfluous unlessmade functional

ProblemThe site provides instructions for walking, bicycling, and driving to, from, and between public transportationstops. These instructions are vague and do not reflect the actual path the user may need to travel. Oneinstruction is given (i.e., “Bike north from 95th St,” or “Drive south on Archer Ave.”), when several actionswould actually be required.

9

Rational1) Match between system and real world: Often times travelers may choose to walk, bicycle, or drive to alocation, or may need to travel to, from, and between stops and stations. The site provides instructions thatdo not match the real world paths of travel.2) Aesthetic and minimalist design: Because many of the directions are not functional, these directions (thenonfunctional instructions) serve only to take up space. The nonfunctional directions are unnecessaryunless they can provide real value for the user,

Examples of appropriate adherence to heuristics Icons match real world concepts of modes of transportation Transit tracker and scheduler removes additional content so only pertinent information is available Condition alerts are only displayed if they affect travel via relevant routes

RecommendationsWalking/Bicycling/driving instructions need to reflect the actual path the user will need to take, or

should be removed.

6. Cancel button appears grayed out when editing trip details


6 Cancel button appears grayed outwhen editing trip details.

3 3 Looks like requirements for cancelhaven’t been met, User is lead to believethat they cannot cancel Edit Trip

ProblemAfter the user has queried the system for directions, the user can edit the query. When the Edit Trip overlayis displayed the cancel button is grayed out.

10

Rational1) Visibility of system status: Suggest that the button is inactive, when it is actually active. Suggests thatthere are actions that must be taken before the button becomes active.2) User control and freedom: A grayed out cancel button on the edit trip overlay will mislead the user toexpect that they cannot exit the edited trip, and that they are required to move forward in the task flow.

Examples of appropriate adherence to heuristics Home page form allows the user to “Show Advanced Options” and then “Hide Advanced Options” The user can manage and customize their preferred routes Transit Tracker and Schedule pages contain buttons that are deactivated until appropriate

information has been supplied

RecommendationsAdd hue to the cancel button.

7. Using Browser navigation causes errors in results


7 Using browser navigation causesproblems for results page

3 7 4 Punishes the user for using interface inan unexpected way, User cannot easilyback navigate without causing errors

ProblemWhen the user navigates between pages using the browser back/forward arrows, the results on the tripresults page cease to display correctly. After navigating back to the results page, the user often no longerable to view the results properly.

Rational1) User control and freedom: The system suffers from errors and punishes the user for using browsercontrols to return to the results page.2) Flexibility and efficiency of use: The rigid controls require the user to requery the site if the user hasnavigated away from the results page.

Examples of appropriate adherence to heuristics Users can input an address in a text field or the user can select from their recent locations from a

dropdown menu Users can control how they receive service alerts

RecommendationsMake results persistent to ensure that users can navigate between the pages using browser

controls without losing previous states

11

8. inconsistencies between controls and constraints


8 Inconsistencies between querybuttons and user constraints

4 3 different button conventions are used indifferent areas of the site

ProblemOn the homepage the button labeled “take me there” is active throughout the task (adding trip information),however if the trip information is not entered correctly then a javascript alert is displayed. The alert requeststhat the user supply required information. On the Transit Tracker and Schedule pages the button thatmoves the user forward in the task flow is deactivated until the appropriate information is selected. The twomodes of interaction are inconsistent with each other.

Rational1) Consistency and standards: There are two conventions for buttons that move the task flow forward. Onebutton (Homepage) is active at all times and uses an alert to correct the user, while two buttons (on theTracker and Schedule pages) are activated only when appropriate information is provided and these buttonsprevent user errors. The three buttons should work similarly because they have the same function anddesign.

Examples of appropriate adherence to heuristics The three buttons used to query the system all share the same design adhering to consistency Page layout is the same for each page where possible

RecommendationsThe forward moving button should become active (acquire orange color fill) only when the user has

provided all information needed to build a trip. Otherwise it should remain greyed out to prevent a user error.

9. Sign out option is only accessible inside the user’s account


9 Sign out option is only accessibleinside the user’s account

4 7 4 Does not follow common conventions,user can sign out from only one location,slows down ability to carry out goal

ProblemThe option to sign out of “My Account” can only be found on a side panel when the user has opened the“My Account” page.

12

Rational1) Consistency and standards: Web sites often place login/signout options in a static location accessiblefrom all pages. Goroo.com breaks this convention by placing the sign out option in an isolated loacation onone page.2) Flexibility and efficiency of use: There is only one location where the user can sign out of his or heraccount. The user must find the option by remembering what page it is on or searching for it. The task ofsigning out is inefficient because the user must access a specific page.

Examples of appropriate adherence to heuristics Language options are consistently present on each page Current time is displayed in upper right hand corner similar to other sites and applications

RecommendationsThere should be a quick link ‘Sign Out” when a user is logged in. That will allow for oneclick sign

out, and it would also be a visual cue that the user is currently signed in and has an option to sign out.

10. No cancel button when updating account


10 No Cancel button when updatingaccount

3 4 The user is not provided with the controlto back out of editing an account.

ProblemThere is no cancel button for the “”Edit MyAccount” page. Once the user has enteredthis screen they are told that their email andpassword are required to move forward(update). There is only an option to update.

13

Rational1) User control and freedom: The user should be free to back out of this editing their account without havingto make any changes. The form suggests that once opened, the “Edit My Account” page requires the user’semail and password and that the user cannot cancel out of the option.

Examples of appropriate adherence to heuristics A cancel button is provided in the “Edit Trip” state Drop down calendar on the Home state has a close button which allows the user to close the

calendar without selecting a date

RecommendationsAdd a cancel button to the “Edit My Account” state that will return the user to “Preferred Routes.”

14

Cognitive Walkthroughs

For the cognitive walkthrough we chose two user tasks: (1) build a trip, and (2) edit your trip.The following tables break down user steps when performing these tasks, and also evaluate how supportivethe system is to the user.

Task 1. User task is obtain public transportation directions from one location to another.

Actions Will users even realizestep is needed?

Will users identifyrelevant control?

Will users recognizeconnectionbetween controland desiredresults?

Step 1 To goroo.com viasearch engine

Yes this a commonpractice

Yes depending on searchengine user may typename in url field or searchfield

Yes standardprocess

Step 2 onclick focus in"From" text field

Yes There is a logicaltask flow for the input ofinformation. The "From"field is placed before the"To" text field.

Yes Text fields in forms isa common practice. Ablinking curser is anindication that an entry isrequired.

Yes The form usesHints shown asexamples

Step 3 Enter address "4653N Beacon St,Chicago, Il 60640"

Yes Text fields in formsis a common practice. Ablinking curser is anindication that an entry isrequired.

The text field has anappropriate length for theentry of an address.

Yes The form is astandard form and iscommonly used.

Step 4 onclick focus in "To"field

Yes There is a logicaltask flow for the input ofinformation. The "To"follows the "From" textfield.

Yes Text fields in forms isa common practice. Ablinking curser is anindication that an entry isrequired.

Recommendation: Afterthe "From" field is enteredauto focus to the "To" textfield.

Yes The form usesHints shown asexamples

Step 5 Enter address "1 EWacker Dr,Chicago, IL 60601

Yes Text fields in formsis a common practice. Ablinking curser is anindication that an entry isrequired.

The text field has anappropriate length for theentry of an address.

Yes The form is astandard form and iscommonly used.

Step 6 Press "Arrive on"radio button

Maybe The task flow islogical but this step in theform does not stand outand the radio button for"Leave" is preselected.

Recommendation Create a section for

Yes radio buttons are useto restrict the user to oneselection "Leave" or "Arriveon".

Yes intuitive

15

departure date and timethat can be differentiatedfrom the rest of the form.

Step 7 Click on calendaricon to select date toarrive

Maybe The calendaricon is placed to the rightof the radio buttons"Leave" and "Arrive on" .The date field isprepopulated and ifoverlooked the systemdefault is the present day.

RecommendationSomehow call attentionthe calendar icon and/ortext field.

Yes Calendar iconcreates a correlationbetween the real world andsystem.

Yes standard iconand used widely

Step 8 Select April 22, 2013on the calendar

Yes Once the calendaris opened the presentdate is selected but asyou hover over otherdates they becomehighlighted alerting theuser that they may makea selection.

Yes Arrows allow usersto select previous or futuremonths. There is also a"close" button to exit thecalendar

Yes calendarplugins are standardto similar applications

Step 9 Click on dropdownarrow to select hourof Arrival

Maybe It follows thecalendar icon.

Recommendation Bring the dropdown menuin to focus.

Yes Dropdowns arecommon practice andintuitive. Clicking on arrowswill allow them to open thedropdown menu.

Yes Dropdownmenus allow users toselect frompreentered data

Step 10 Select the number"2" to indicate thetwo o'clock hour

Yes Dropdown menusallow users to select frompreentered data

Yes intuitive Yes intuitive

Step 11 Click on dropdownarrow to select theminute of Arrival

"Maybe It follows thehour dropdown menu.

Recommendation Bringthe dropdown menu in tofocus. "


Yes Dropdownmenus allow users toselect frompreentered data

Step 12 Select the number"30"

Yes Dropdown menusallow users to select frompreentered data

Yes intuitive Yes intuitive

Step 13 Select pm fromdropdown menu

Maybe Dropdownmenus allow users toselect from preentereddata


Yes Dropdownmenus allow users toselect frompreentered data.

Step 14 make sure "I preferaccessibleservices" "No" radiobutton is selected

Maybe "No" button isselected by default

Recommendation Makethe layout in order steps

Yes Radio buttons areused and only allow theuser to select one option

Yes There is anicon to assist users

16

Step 15 Select fromtransportation modepreferences "Train"checkbox

Maybe multiplecheckboxes are selected.User may forget todeselect boxes that theydo not need.

Yes Check boxes arestandard a

Yes The text is selfexplanatory

Task 2. User would like to modify the trip: get reverse directions to get home at 8 pm that day, and tocheck how they can get to their destination by other modes of transportation (bus).

Actions Will users even realizestep is needed?

Will users identify relevantcontrol?

Will users recognizeconnection betweencontrol and desiredresults?

Step 1 Find and click on"Edit Your Trip"

Maybe users maynot be aware of anoption to updatecurrent trip with noneed to go to theHomepage and rebuilda trip

Maybe the link is somewhathidden, not very noticeable onthe page. Or the wording of thelink does not provide enoughinformation about what's behindthat link.

Recommendation: bring thisfeature into focus; make it abutton and/or add an icon tomake this option easy to locateon the page. Also, provideenough text in the link/buttonthat would clearly communicatewhat is behind the link (e.g. Editor Modify Your Trip").

Yes the link (anunderlined text) is astandard way to getto another page

Step 2 Locate and clickon the 'Reversedirections' buttonbetween 'From'and 'To' text fields

Yes this step islogical and matchesthe system and realworld

Maybe the iconrepresentation may not beintuitive to all users. Also, inwhen hovering over the iconwith a mouse the system doesnot provide a description of thisUI element.

Recommendation: add textlabel to the icon and tagdescription

Maybe some usersmay not understandthat this UI elementwill reverse the"From" and "To"addresses

Step 3 Click on dropdownarrow to selecthour of arrival

Yes users arefamiliar with this stepwhen they first builtthis trip

Yes users have already usedthis control before

Yes standard formto expand availableoptions

Step 4 Select number "8"to indicate thethree o'clock hour



Yes standard wayto select an item

17

Step 5 Click on dropdown arrow toselect minutes ofarrival



Yes standard wayto expand availableoptions

Step 6 Click on "00" toindicate zerominutes



Yes standard wayto select an item

Step 7 SelectTransportationMode Preferencesby clicking 'Bus'checkbox



Yes standard wayto select multipleitems

Step 8 Click on "UpdateTrip" button

Yes in order tocomplete the tripupdates and get newdirection results page,users need to clickthis button

Yes the button representationis a standard form, and thebutton wording matches thetask in hand

Yes intuitive

Review of Top Priority Recommendations

We conducted heuristics analysis of the site and identified some major design and functionality problems ofthe site. Cognitive walkthrough of two user tasks helped us evaluate how the system supports user mentalmodel when building or modifying a trip. Based on our findings we suggest three top priorityrecommendations to improve usability. We gave these recommendations priority due to the severity ofproblems they cause the user.

Recommendation 1.Make results persistent to ensure that users can navigate between the pages using browser controlswithout losing previous states.

Recommendation 2.Walking/Bicycling/driving instructions need to reflect the actual path the user will need to take, or should beremoved.

Recommendation 3.Account set up should display the email notification in an overly window instead of a javascript alert. Focusof the confirmation should be prominent, and include colors and icons that would clearly communicate thatthe user has successfully completed a signup (e.g. check mark, green colors). And also notify that anemail was sent to the provided email address.

18

project 1. heuristic evaluation and cognitive walkthrough ... · this report is an analysis of...

Documents