streamlined product evaluation
DESCRIPTION
When you have limited resources and no access to users, this process will allow you to evaluate a product's usability and present findings and recommendations to the product team.TRANSCRIPT
1
Catherine WeeksSeaChange International
2
3
& Product Evaluation
“This is a more specific version of a design walkthrough, focusing on cognitive principles.Based on a user's goals, a group of evaluators steps through tasks, evaluating at each step how difficult it is for the user to identify and operate the interface element most relevant to their current subgoal and how clearly the system provides feedback to that action. Cognitive walkthroughs take into consideration the user's thought processes that contribute to decision making, such as memory load and ability to reason.”– Usability First (http://www.usabilityfirst.com/methods/cogwalk.txl)
Expert Review
Heuristic Review
Cognitive Walkthrough
“Heuristic evaluations usually are conducted by a small set (one to three) of evaluators. The evaluators independently examine a user interface and judge its compliance with a set of usability principles. The result of this analysis is a list of potential usability issues or problems. The usability principles, also referred to as usability heuristics, are taken from published lists. Ideally, each potential usability problem is assigned to one or more heuristics to help facilitate fixing the problem. As more evaluators are involved, more true problems are found.” – Usability.gov (http://www.usability.gov/methods/heuristiceval.html)
“Expert usability reviews are a fast and economical route to improving the usability of a Web site... The report outlines the criteria used to evaluate the site with reliable recommendations, which are based on extensive Human-Computer Interaction (HCI) research and experience.” – Mich. State Usability & Accessibility Center (http://usability.msu.edu/services_expertRev.asp)
4
Expert Review
Experience Research Many evaluators
Heuristic Review
Principles Non-expert Comparable
Cognitive Walkthrough
Task centric Working prototype Timely
5
•Limited or no product team access•Limited UX resources•Limited time•No access to users
•Access to working applications
This led me to create a process based on a twist of the product evaluation types…
6
Identify User
Product Review
Gives 1,000 foot view Looks at branding, product placement, end user satisfaction
Cognitive Walkthrough
Look through application based on user tasks Find interaction problems, layout issues, and even bugs!
Create Recommendations
Produce Report
7
Case Intro:N
o team"
Pet" projectN
o requirements, no business caseN
atural extension to current products
Identify User:T
echnicalB
usiness solution
8
Product Review Findings
“Customers will not be able to use [Product name] outside of [Parent product name], therefore creating a barrier to use…”
“…providers could alter the recommendations without any indication to the user that these are ‘sponsored’ rather than purely calculated recommendations.”
Cognitive Walkthrough Findings
If the user selects the incorrect operation, in order to fix their mistake, the user must delete the new rule, and add another rule…
The colors do not support color blindness (red and green used)
9
Case Intro:
Offshore office (Asia)
Customer buy-in
Responsive team
Identify User:
Technical
Business solution
10
Product Review Findings
It is unclear where the user should begin in order to make the system work, and would require a vast amount of training in order for the users to become proficient in the system.
Correct SeaChange product branding was not used.
Cognitive Walkthrough Findings
It is unclear how to access this application when receiving the hardware.
The content area pane is laid out well and supports the user’s work. The whole page or continuing page look helps users identify if they need to scroll to see more information.
11
Requires few resources
Can be as minimal or extensive as needed
Produces a report that is easy for the product team to work with
12
Relies on experience and research, not user input!
Reliance on few people to catch many issues
Requires understanding of business needs
Limited team collaboration opportunities
13
Access to a working interface or prototype
Product Review: 1 ~ 3 daysCognitive Walkthrough: 3 ~ 5 daysRecommendations: 2 ~ 5 daysReport: 2 ~ 4 days
Total: 2 ~ 4 weeks
14
Product is perceived as valuable to company strategy
Limited resources
No access to users
15
Works well in limited resource situations
Can provide minimal to extensive reviews
Should always opt to have users involved if possible
Results in a clear report for the product team
16
OK/Cancel Comics:
http://www.ok-cancel.com/comic/18.html, http://www.ok-cancel.com/comic/82.html
Cognitive Walkthrough definition: http://www.usabilityfirst.com/methods/cogwalk.txl
Heuristic Review definition:
http://www.usability.gov/methods/heuristiceval.html
Expert Review definition:
http://usability.msu.edu/services_expertRev.asp
18
19