exploring usability testing for mobile and web technologies
TRANSCRIPT
TJ AM Tutorial
10/14/2014 8:30:00 AM
"Exploring Usability Testing for Mobile
and Web Technologies"
Presented by:
Rob Sabourin
AmiBug.com
Brought to you by:
340 Corporate Way, Suite 300, Orange Park, FL 32073
888-268-8770 ∙ 904-278-0524 ∙ [email protected] ∙ www.sqe.com
Rob Sabourin
AmiBug.com Rob Sabourin, P. Eng., has more than thirty years of management experience leading teams of software development professionals. A well-respected member of the software engineering community, Rob has managed, trained, mentored, and coached hundreds of top professionals in the field. He frequently speaks at conferences and writes on software engineering, SQA, testing, management, and internationalization. Rob wrote I am a Bug!, the popular software testing children's book; works as an adjunct professor of software engineering at McGill University; and serves as the principle consultant (and president/janitor) of AmiBug.Com, Inc. Contact Rob [email protected].
© 2013 Robert Sabourin 1
© 2013 Robert Sabourin EUT v1.1 1
Exploring Usability Testing
Robert Sabourin
President
AmiBug.Com, Inc.
Montreal, Canada
WELCOME
Exploring Usability Testing
© 2013 Robert Sabourin EUT v1.1 2
© 2013 Robert Sabourin 2
• Robert Sabourin ,
Software Evangelist
• President
• AmiBug.Com Inc.
• Montreal, Quebec,
Canada
• www.amibugshare.com
Exploring Usability Testing
© 2013 Robert Sabourin EUT v1.1 3
Exploring Usability Testing
Exploring Usability Testing
•Net Promoter Score
•System Usability Scale
•Exploratory Testing
•Whiteboarding
•Usability Heuristics
© 2013 Robert Sabourin EUT v1.1 4
© 2013 Robert Sabourin 3
Exploring Usability Testing
Exploring Usability Testing
•Forms and Workflows
•Usability Reviews
•Do It Yourself Usability Testing
•Some Mobility Usability Concerns
•Some Accessibility Concerns
© 2013 Robert Sabourin EUT v1.1 5
NON-FUNCTIONAL TESTING
Exploring Usability Testing
© 2013 Robert Sabourin EUT v1.1 6
© 2013 Robert Sabourin 4
Non-Functional Testing
Quality Factors
Attributes
Characteristics
Reliability
Other “-ilities”
© 2013 Robert Sabourin EUT v1.1 7
Non-Functional Testing Challenges
Goals
• Elicit
• Articulate
• Quantify
Subjective
• Needs or wants?
• Relative or absolute?
• What is good enough?
Tests
• Tricky to orchestrate
• Difficult to interpret results
• Challenging to baseline or regress
8© 2013 Robert Sabourin EUT v1.1
© 2013 Robert Sabourin 5
Non-Functional Agile Challenges
Time
• Analysis
• Model
• Prepare
• Run
• Interpret
Change
• Baselines
• Goals
Software
• Continuous integration
• Incomplete product builds
9© 2013 Robert Sabourin EUT v1.1
Non-Functional Testing
Principles
• Seek to understand why attributes are valued by stakeholders
• Seek to discover relationship between non-functional test and value
• Use multiple tools and techniques
• Use heuristic guides
© 2013 Robert Sabourin EUT v1.1 10
© 2013 Robert Sabourin 6
ISO 9126 Quality Factors
•Attributes of the effort needed for use, and on the individual assessment of such use, by a set of users
•Learnability
•Understandibility
•Operability
Usability
© 2013 Robert Sabourin EUT v1.1 11
Quality Factors
© 2013 Robert Sabourin EUT v1.1 12
© 2013 Robert Sabourin 7
Product Backlog Constraints
Tom Gilb, Competitive Engineering, suggests defining
Scale: "What is measured"
Meter: "How to measure (method)"
Target: "Level we're aiming for. Success"
Constraint: "Level we're seeking to avoid. Failure"
Benchmark: "Where we are today"
13© 2013 Robert Sabourin EUT v1.1
Non-Functional Testing
14© 2013 Robert Sabourin EUT v1.1
© 2013 Robert Sabourin 8
Non-Functional Experimentation• Iterative
• Implement trial
• Measures result
Timebox non-functional tests
• Pause
• Review findingsAfter timebox
• Investigate more
• Refactor goal
• Move on to something elseNext steps
15© 2013 Robert Sabourin EUT v1.1
Experimenting
Conjecture
ConfirmationRefutation
© 2013 Robert Sabourin EUT v1.1 16
© 2013 Robert Sabourin 9
Non-Functional Exploratory Testing
Usability
Pair with operator
Pair with domain experts
Pair with human factors guru
© 2013 Robert Sabourin EUT v1.1 17
NET PROMOTER SCORE
Exploring Usability Testing
© 2013 Robert Sabourin EUT v1.1 18
© 2013 Robert Sabourin 10
Net Promoter Score
© 2013 Robert Sabourin EUT v1.1 19
Net Promoter Score
© 2013 Robert Sabourin EUT v1.1 20
© 2013 Robert Sabourin 11
Net Promoter Score
•11 values
•0 not at all likely
•10 extremely likely
Response Scale
© 2013 Robert Sabourin EUT v1.1 21
Net Promoter Score
• respondents giving a 9 or 10 score
Promoters
• respondents giving a 7 or 8 score
Passives
• respondents giving a 0 to 6 score
Detractors
© 2013 Robert Sabourin EUT v1.1 22
© 2013 Robert Sabourin 12
Net Promoter Score
•The difference between the percentage of Promoters and Detractors
• Is not expressed as a percentage
• Is an absolute number lying between -100 and +100
•uTest suggests “any positive number is good” (ref: TK Maxx Mobile Usability Testing Results Report)
NPS
© 2013 Robert Sabourin EUT v1.1 23
Net Promoter Score
© 2013 Robert Sabourin EUT v1.1 24
© 2013 Robert Sabourin 13
© 2013 Robert Sabourin EUT v1.1 25
SYSTEM USABILITY SCALE
Exploring Usability Testing
© 2013 Robert Sabourin EUT v1.1 26
© 2013 Robert Sabourin 14
System Usability Scale
•Measures usability in terms of effectiveness, efficiency and satisfaction
• SUS can be read as a percentage
• SUS can be used as a benchmarking mechanism
• SUS can be used to compare the current system to competitors’ systems
• SUS can be used to compare the current system to future versions
SUS - System Usability Scale
© 2013 Robert Sabourin EUT v1.1 27
System Usability Scale
© 2013 Robert Sabourin EUT v1.1 28
© 2013 Robert Sabourin 15
System Usability Scale
1. I think that I would like to use this system frequently2. I found the system unnecessarily complex3. I thought the system was easy to use4. I think that I would need the support of a technical person to be able to use this system
5. I found the various functions in this system were well integrated
6. I thought there was too much inconsistency in this system
7. I would imagine that most people would learn to use this system very quickly
8. I found the system very cumbersome to use9. I felt very confident using the system10. I needed to learn a lot of things before I could get going with this system
© 2013 Robert Sabourin EUT v1.1 29
SUS Questions
System Usability Scale
© 2013 Robert Sabourin EUT v1.1 30
SUS Responses
© 2013 Robert Sabourin 16
System Usability Scale
• For odd items: subtract one from the user response.
• For even-numbered items: subtract the user responses from 5
• This scales all values from 0 to 4 (with four being the most positive response).
• Add up the converted responses for each user and multiply that total by 2.5. This converts the range of possible values from 0 to 100 instead of from 0 to 40.
© 2013 Robert Sabourin EUT v1.1 31
Scoring SUS
System Usability Scale
© 2013 Robert Sabourin EUT v1.1 32
Scoring SUS
© 2013 Robert Sabourin 17
EXPLORATORY TESTING
Exploring Usability Testing
© 2013 Robert Sabourin EUT v1.1 33
Exploratory Testing
© 2013 Robert Sabourin EUT v1.1
Learning
DesignExecution
34
© 2013 Robert Sabourin 18
Charter Statement
• Statement of mission
• Ties to purpose
• Focuses work
• Confirms understanding
• Delineates scope
• Analogy to test story
© 2013 Robert Sabourin EUT v1.1 35
Charter Statement
• Short, terse
• To the point
• Inclusions '
• Exclusions '
• Limits '
© 2013 Robert Sabourin EUT v1.1 36
© 2013 Robert Sabourin 19
eBay Usability Charters
© 2013 Robert Sabourin EUT v1.1 37
eBay Usability Charters
© 2013 Robert Sabourin EUT v1.1 38
© 2013 Robert Sabourin 20
eBay Usability Charters
© 2013 Robert Sabourin EUT v1.1 39
eBay Usability Charters
© 2013 Robert Sabourin EUT v1.1 40
© 2013 Robert Sabourin 21
Pairing Testers With …
Non Functional Testing
•Usability Gurus
•SMEs
•Human Factors Experts
© 2013 Robert Sabourin EUT v1.1 41
WHITEBOARDING
Exploring Usability Testing
© 2013 Robert Sabourin EUT v1.1 42
© 2013 Robert Sabourin 22
Prototype Styles
• Paper and pencil• Comic books style story boards• Wire frame models• Mock ups• Prototypes• Site maps• User flow• Wire flow• Story boards• Wire frame• Page description diagram• Functional specification
© 2013 Robert Sabourin EUT v1.1 43
Whiteboarding
© 2013 Robert Sabourin EUT v1.1 44
© 2013 Robert Sabourin 23
Whiteboarding
© 2013 Robert Sabourin EUT v1.1 45
Whiteboarding
© 2013 Robert Sabourin EUT v1.1 46
© 2013 Robert Sabourin 24
Whiteboarding
What is Whiteboarding?
Collaboration
Coordination
Conceptualization
© 2013 Robert Sabourin EUT v1.1 47
Tony Buzan
Mind Maps
© 2013 Robert Sabourin EUT v1.1 48
© 2013 Robert Sabourin 25
Whiteboarding
© 2013 Robert Sabourin EUT v1.1 49
Whiteboarding
User Mind Map
© 2013 Robert Sabourin EUT v1.1 50
© 2013 Robert Sabourin 26
Buying a Book
Usage Scenarios
© 2013 Robert Sabourin EUT v1.1 51
Whiteboarding
Story Board
© 2013 Robert Sabourin EUT v1.1 52
© 2013 Robert Sabourin 27
Wrap-O-Matic
Usage Scenarios
© 2013 Robert Sabourin EUT v1.1 53
Scenario Based Testing
© 2013 Robert Sabourin EUT v1.1 54
© 2013 Robert Sabourin 28
Buying a Book
Usage Scenarios
© 2013 Robert Sabourin EUT v1.1 55
© 2013 Robert Sabourin EUT v1.1
Slide 56
© 2013 Robert Sabourin 29
© 2013 Robert Sabourin EUT v1.1
Slide 57
© 2013 Robert Sabourin EUT v1.1 58
© 2013 Robert Sabourin 30
Social Networking
Usage Scenarios
© 2013 Robert Sabourin EUT v1.1 59
© 2013 Robert Sabourin EUT v1.1 60
© 2013 Robert Sabourin 31
Scenario Based Testing
© 2013 Robert Sabourin EUT v1.1 61
Scenario Based Testing
© 2013 Robert Sabourin EUT v1.1 62
© 2013 Robert Sabourin 32
Scenario Based Testing
© 2013 Robert Sabourin EUT v1.1 63
Scenario Based Testing
© 2013 Robert Sabourin EUT v1.1 64
© 2013 Robert Sabourin 33
Scenario Based Testing
© 2013 Robert Sabourin EUT v1.1 65
Scenario Based Testing
© 2013 Robert Sabourin EUT v1.1 66
© 2013 Robert Sabourin 34
Scenario Based Testing
© 2013 Robert Sabourin EUT v1.1 67
Scenario Based Testing
© 2013 Robert Sabourin EUT v1.1 68
© 2013 Robert Sabourin 35
USABILITY HEURISTICS
Exploring Usability Testing
© 2013 Robert Sabourin EUT v1.1 69
Jakob Nielsen
• Usability heuristics
• Rules of thumb
• General principles of user interface design
• www.nngroup.com
© 2013 Robert Sabourin EUT v1.1 70
© 2013 Robert Sabourin 36
Usability Heuristics
• Visibility of system status
– Always keep users informed
– Provide appropriate feedback
– Respond in a reasonable timeframe
© 2013 Robert Sabourin EUT v1.1 71
Usability Heuristics
• Match between system and the real world
– Speak the users' language
– Use familiar concepts
– Follow a natural order
© 2013 Robert Sabourin EUT v1.1 72
© 2013 Robert Sabourin 37
Usability Heuristics
• User control and freedom
– Provide means to exit from unwanted states
– Provide undo and redo capabilities
© 2013 Robert Sabourin EUT v1.1 73
Usability Heuristics
• Consistency and standards
– Same words mean same thing in different contexts
– Follow environment conventions
© 2013 Robert Sabourin EUT v1.1 74
© 2013 Robert Sabourin 38
Usability Heuristics
• Error prevention
– Prevents problems from occurring
– Eliminate error-prone conditions
– Have a confirmation option
© 2013 Robert Sabourin EUT v1.1 75
Usability Heuristics
• Recognition rather than recall
– Making objects visible
– Make instructions available
© 2013 Robert Sabourin EUT v1.1 76
© 2013 Robert Sabourin 39
Usability Heuristics
• Flexibility and efficiency of use
– Accelerators for the expert user
– Make common options adaptable
© 2013 Robert Sabourin EUT v1.1 77
Usability Heuristics
• Aesthetic and minimalist design
– Dialogues should contain relevant information
© 2013 Robert Sabourin EUT v1.1 78
© 2013 Robert Sabourin 40
Usability Heuristics
• Help users recognize, diagnose, and recover from errors
– Use plain language for error messages
– Constructively suggest solutions
© 2013 Robert Sabourin EUT v1.1 79
Usability Heuristics
• Help and documentation
– Easy to search
– Focused on the user's task
– Offer concrete steps to be carried out
– Short and to the point
© 2013 Robert Sabourin EUT v1.1 80
© 2013 Robert Sabourin 41
USABILITY SEVERITY LEVELS
Exploring Usability Testing
© 2013 Robert Sabourin EUT v1.1 81
Usability Severity
© 2013 Robert Sabourin EUT v1.1 82
© 2013 Robert Sabourin 42
ISO 9421
Exploring Usability Testing
© 2013 Robert Sabourin EUT v1.1 83
Usability ISO 9421
•the extent to which the intended goals of use are achieved
Effectiveness
•the resources that have been expended to achieve the intended goals
Efficiency
•the extent to which the user finds the use of the product acceptable
Satisfaction
•can user be harmed
Safety
© 2013 Robert Sabourin EUT v1.1 84
© 2013 Robert Sabourin 43
Usability ISO 9421
•the extent to which the system is complete in design and meets its desired goals
Completeness
•Is the capability of a software product to enable the user to learn how to use it
Learn-ability
•the extent to which the product fits into normal course of daily life
Convenience
•Confusing or vague usability. Misleading information that might also result in providing no usability at all
Ambiguity
© 2013 Robert Sabourin EUT v1.1 85
FORMS AND WORKFLOWS
Exploring Usability Testing
© 2013 Robert Sabourin EUT v1.1 86
© 2013 Robert Sabourin 44
Form Design Principles
• Minimize pain• Illuminate path• Consider context• Ensure consistent communication
© 2013 Robert Sabourin EUT v1.1 87
Luke Wroblewski
Web Form Design
Eye Tracking
© 2013 Robert Sabourin EUT v1.1 88
© 2013 Robert Sabourin 45
Form Usability
• Number and location of errors• Severity of errors• Completion rates• Time to complete forms• Satisfaction scores• Subjective comments
© 2013 Robert Sabourin EUT v1.1 89
Awkward Form Path
© 2013 Robert Sabourin EUT v1.1 90
© 2013 Robert Sabourin 46
Clear Form Path
© 2013 Robert Sabourin EUT v1.1 91
Form Usability
• Eye tracking– What people looked at
– Number of eye fixations
– Length of eye fixations
© 2013 Robert Sabourin EUT v1.1 92
© 2013 Robert Sabourin 47
Form Usability Checklist
�Any unnecessary questions�Succinct questions�Sincere voice�Natural language�Logical groups of information�Structure form as a conversation�Ask optional questions separately
© 2013 Robert Sabourin EUT v1.1 93
Form Usability Checklist
�Name the form�Name sections of the form�Clear scan line�Spacing between sections�Minimal distractions�Start page guidance for long forms�Indicate progress
© 2013 Robert Sabourin EUT v1.1 94
© 2013 Robert Sabourin 48
Progress Indication
© 2013 Robert Sabourin EUT v1.1 95
Input Validation
�Suggest valid answers�Illustrate valid format�Validate syntax after user enters data�Indicate limits or ranges of inputs�Provide smart defaults�Hide unneeded controls
© 2013 Robert Sabourin EUT v1.1 96
© 2013 Robert Sabourin 49
Error Messages
�Provide clear error messages�Indicate if error blocks completion�Guide users to resolve error�Provide visual emphasis of error�Use red text and icons for errors�Indicate successful completion
© 2013 Robert Sabourin EUT v1.1 97
DO IT YOURSELF USABILITY TESTING
Exploring Usability Testing
© 2013 Robert Sabourin EUT v1.1 98
© 2013 Robert Sabourin 50
Do it yourself
© 2013 Robert Sabourin EUT v1.1 99
Do it yourself
A morning a month
Get insights from users
Watch real users
© 2013 Robert Sabourin EUT v1.1 100
© 2013 Robert Sabourin 51
Do it yourself
Recruit diversely
Find important problems
Improve design
© 2013 Robert Sabourin EUT v1.1 101
Do it yourself
Three testers
Test on site
Observe shared screens
© 2013 Robert Sabourin EUT v1.1 102
© 2013 Robert Sabourin 52
Do it yourself
Observers take notes
User questions recorded
Action decided at lunch
© 2013 Robert Sabourin EUT v1.1 103
SOME MOBILE USABILITY CONCERNS
Exploring Usability Testing
© 2013 Robert Sabourin EUT v1.1 104
© 2013 Robert Sabourin 53
Mobile Usability
•Small Screens
•Limited bandwidth
•Fingers
•No Flash Like GUI
Difficulties
© 2013 Robert Sabourin EUT v1.1 105
Mobile Usability
•Bad first impression
•Operator errors
•Negative feedback / reviews
•Competitor fills gap
Risks
© 2013 Robert Sabourin EUT v1.1 106
© 2013 Robert Sabourin 54
Mobile Usability
• Important stuff on top
• Easy to find stuff
• Single column layout
• Minimize navigation
• Light data
Guidelines
© 2013 Robert Sabourin EUT v1.1 107
Mobile Usability
• Finger taps
• Avoid fancy web gui technologies
• Minimize forms
• Minimize feature sets
• Separate mobile web resources
• Fast
Guidelines
© 2013 Robert Sabourin EUT v1.1 108
© 2013 Robert Sabourin 55
Mobile Usability
•In the lab
•Beta
•Crowd source
Mobile Usability Test
© 2013 Robert Sabourin EUT v1.1 109
Mobile Usability
• Vary and control
• Technologically
• OS
• Browser
• Device
• Carrier
• Bandwidth
Mobile Usability Test
© 2013 Robert Sabourin EUT v1.1 110
© 2013 Robert Sabourin 56
Mobile Usability
•Geographically
• Continent
• Country
•City
• Language
• Locale
Vary and control
© 2013 Robert Sabourin EUT v1.1 111
Mobile Usability
•Demographically
•Age
•Gender
• Education
•Employment
• Industry
•Computer savvy
•Domain expertise
Vary and control
© 2013 Robert Sabourin EUT v1.1 112
© 2013 Robert Sabourin 57
SOME ACCESSIBILITY CONCERNS
Exploring Usability Testing
© 2013 Robert Sabourin EUT v1.1 113
Accessibility
Does application support
•Special needs of end user
•Disabilities of end user
•Older end users
•End users in rural areas
•End users in developing countries
© 2013 Robert Sabourin EUT v1.1 114
© 2013 Robert Sabourin 58
Accessibility
• Difficult to see
• Difficult to hear
• Difficult to interpret text
• Difficult to point to object
• Difficult to use keyboard
• Difficult to use mouse
Some Examples
© 2013 Robert Sabourin EUT v1.1 115
Accessibility
Tools support http://www.w3.org/WAI/ER/tools/complete
• Bobby (version 5.3)Watchfire, 14 March 2005Description: Watchfire® Bobby™ 5.0 is a web accessibility desktop testing tool designed to help expose barriers to accessibility and encourage compliance with existing accessibility guidelines, including Section 508 of the U.S. Rehabilitation Act and the W3C's Web Content Accessibility Guidelines (WCAG). Bobby spiders through a website and tests to see if it meets accessibility requirements, including readability by screen readers, the provision of text equivalents for all images, animated elements and audio and video displays. During a scan, Bobby checks HTML against select accessibility guidelines and then reports on the accessibility of each web page.Guidelines: WCAG 1.0, Section 508Automatic checking: Single pages, Page groups, Restricted pagesFormats: HTML, XHTML
© 2013 Robert Sabourin EUT v1.1 116
© 2013 Robert Sabourin 59
Accessibility
Accessibility Failures
• http://www.w3.org/standards/webdesign/accessibility
• Taxonomy of example accessibility failures
© 2013 Robert Sabourin EUT v1.1 117
Accessibility
•http://www.color-blindness.com/coblis-color-blindness-simulator/
• http://www.vischeck.com
Color Blindness Simulator
•http://www.screen-resolution.com/
Resolution Simulator
© 2013 Robert Sabourin EUT v1.1 118
© 2013 Robert Sabourin 60
Bibliography
• Casey, S. (1998), Set Phasers to Stun: And other true tales of design, technology, and humanerror, Agean, Santa Barbara.
• Covey, S. (2000), Living the 7 Habits : The courage to change, Simon & Schuster, New York.
• Crosby, P. B. (1990), Let's Talk Quality: 96 Questions You Always Wanted to Ask Phil Crosby, Plume, New York.
• Crosby, P. B. (1988), The Eternally Successful Organization, Penguin Group, New York.
• Derman, E. (2011), Models.Behaving.Badly: Why confusing illusion with reality can lead to disaster, on wall street and in life, Free Press, New York.
• Dustin, E. (2002), Quality Web Systems : Performance, security, and usability, Addison Wesley, Boston.
• Fine, M. R. (2002), Beta Testing for Better Software, Wiley Technology Publishing, New York.
• Gawande, A. (2009), The Checklist Manifesto, Metropolitan Books, New York.
• Gigerenzer, G. & Todd, P. M. (1999), Simple Heuristics That Make Us Smart, Oxford University Press, Oxford, UK.
• Gilb, T. (2005), Competitive Engineering: A Handbook For Systems Engineering, Requirements Engineering, and Software Engineering Using Planguage, Butterworth-Heinemann, Amsterdam, Netherlands.
© 2013 Robert Sabourin EUT v1.1 119
Bibliography
• Gilb, T. & Graham, D. (1993), Software Inspection, Addison-Wesley, Harlow, England.
• Gilb, T. & Weinberg, G. (1977), Humanized Input: Techniques for Reliable Keyed Input, Winthrop Publishers, Inc., Wellesley, MA.
• Guaspari, J. (1985), I Know It When I See It, Amacon, New York.• indle, T. (1998), Interviewing Skills, Fenn Publishing Company Ltd., Bolton, ON.
• Jonassen, D. H.; Tessmer, M. & Hannum, W. H. (1999), Task Analysis Methods for Instructional Design, Lawrence Erlbaum Associates, Publishers, Mahwah, New Jersey.
• Kaner, S.; Lind, L.; Toldi, C.; Fisk, S. & Berger, D. (1996), Facilitator's Guide to Participatory Decision-Making, New Society Publishers, Gabiola Island, BC.
• Kluger, J. (2009), Simplexity : Why simple things become complex (and how complex things can be made simple), Hyperion, New York.
• Krug, S. (2010), Rocket Surgery Made Easy, New Riders, Berkeley, Califonia.
• Krug, S. (2006), Don't Make Me Think, New Riders, Berkeley, California.
• Norman, D. (1993), Things That Make us Smart : Defending human attributes in the age of the machine, Perseus Books, Reading, Mass.
• Norman, D. A. (1999), The Invisible Computer: Why Good Products Can Fail, the Personal Computer Is So Complex, and Information Appliances Are the Solution, The MIT Press, Cambridge, MA.
© 2013 Robert Sabourin EUT v1.1 120
© 2013 Robert Sabourin 61
Bibliography
• Patton, M. Q. (2002), Qualitative Research & Evaluation Methods, Sage Publications, Inc., Thousand Oaks, CA.
• Roam, D. (2011), Blah Blah Blah : What to do when words don't work, Portfolio/Penguin, New York.
• Roam, D. (2009), Unfolding the Napkin : The hands-on method for solving complex problems with simple pictures, Portfolio, New York.
• Roam, D. (2008), The Back of the Napkin: Solving Problems and Selling Ideas with Pictures, Penguin Books, New York.
• Rosson, M. (2002), Usability Engineering : Scenario-based development of human-computer interaction, Academic Press, San Francisco.
• Stephens, M. (2010), Design Driven Testing : Test smarter, not harder, Apress, New York.
• Surowiecki, J. (2005), The Wisdom of Crowds, Anchor Books, New York.
• Ware, C. (2008), Visual Thinking for Design, Morgan Kaufmann Publishers, Burlington, Massachusetts.
• Carrol, J., ed. (1995), Scenario-Based Design: Envisioning work and technology in system development, John Wiley & Sons, New York.
© 2013 Robert Sabourin EUT v1.1 121