what your customers really think: incorporating usability testing into agile
DESCRIPTION
I did this talk for Agile Africa 2014 You can’t know whether your agile project is maximising is impact unless you gather customer feedback. But the feedback that comes to you is not always the full story. This talk looks at why you should actively go an get user feedback with usability testing, and how to go about doing your first usability test.TRANSCRIPT
What your users really think Incorporating user testing and research into Agile
Phil Barrett • Flow Interactive • Agile Africa 2014
UX design, research and strategy London & Cape Town !Since 1998 !
Hello.
You check out some new digital product.
You“Meh.”
You“Meh.”
Dear sir/madam !I recently visited your new website. I must confess that at first, I found it a bit difficult to understand the value you were offering. !I wasn’t sure if it was because I was not part of your target market, or if you were still working out exactly what was the best thing to offer. !I persevered and after a while realised that you were actually providing a potential useful service. I think it could be extremely profitable if you simply make the following changes: !1.Ensure that I am not required to register before
Not you
Who sends you feedback?
No opinion Tentative opinion Strong opinion
Not using
Trying out
Casual user
Evangelist/Beta group
User
Who sends you feedback?
Feedback
Silence
No opinion Tentative opinion Strong opinion
Not using
Trying out
Casual user
Evangelist/Beta group
User
Missing feedback breaks Agile’s awesomeness
Build
Learn
Measure
Agile projects deliver gradual, incremental change
so it’s easy to miss overall user impact.
Flickr: Lars ploughman
Web analytics, tracking
What do people do with the product? !Hard data.
Split testing
What will people do if we try something else? !Experimentation.
The learn stack
UX testing sessions
Why do people do that? !Causes, inspiration, direction, prediction outside of known situations. !Innovation.
F2F user testing gives you a blast of reality
I don’t need no stinkin’ feedback
• I know what people want
• My software is clearly good. I mean: look!
• Customers don’t know what they want
Flickr: Daniel D
ionne
I know what people want
You’re much better with software than most of your users
Website tasks:the slowest 25% of users take
2.4 timesas long as the fastest 25% of users
Your usersYou
http://www.nngroup.com/articles/variability-in-user-performance/
Econ: Considers, calculates, makes optimal decisions
Human: Emotions, shortcuts and irrationality
Real people are unpredictable
My software is good: look!
Dan Ariely did an experiment.
With origami frogs.
They were hard to make and most people did a bad job.
How much would people bid for their own frogs?
And the frogs of others? And expert -made frogs?
Flickr: Todd Jordan/Tojosan
Flickr: Nanim
o
We think the things we make are expert quality.
• Average bid for expert-made frog: 27¢
• Average bid for own frog: 23¢
• Average bid by someone else for that same frog: ¢5c
Even when they are not.
“
The 9x effect
Executives, overvalue their own innovations...
Companies overweight the new product’s benefits by a factor of
9
John T. Gourville
Harvard Business School
3
Consumers overweight the incumbent product’s benefits by a factor of
3
Customers don’t know what they want
Humans are bad at imagining the future
Understand the jobs to be done
“To design an easy-to-use interface, pay
attention to what users do, not what they say.
!Self-reported claims are unreliable, as are user
speculations about future behaviour.
Jakob Nielsen NNGroup
Use observation
“There is a direct correlation between the number of hours each team member is exposed directly to real users and the improvements we see in the designs. !It's the closest thing we've found to a silver bullet.
Jared Spool UIE
Let’s make it incredibly easy
Doing a usability test
1. Get an interface. List key tasks.
2. Ask someone new to try doing the tasks, and think aloud.
3. Don’t interfere.
Just write thingsdown.
!Wait for 4 seconds.
4. Q. “Is this right?” A. “What do you think?”
!!Be weird but friendly.
5.
Record everything so you and your team can review the issues.
Open questions and storytelling
Do you like this? What do you think?
Do you understand this? What is this for?
Does this annoy you? How does this make you feel?
Do you want this? When will you use this?
Do you usually do this? Tell me the story of the last time you did something like this…
Get users from…
• The next desk • The canteen • Your forums • Market research recruiters
Baby steps: Hall testing
You don’t need working code
Do usability tests in every sprint
Just tell the recruiter to get you “5 users every thursday.”
Evaluate
Implement
Design and analysis Design and analysis
Implement
Design and analysis
Evaluate
Implement
Evaluate Evaluate
Design and analysis
Implement
Test a mix of stuff
Past FuturePresent
Interviews about past
experiences
Testing working software
Testing mockups and concepts
Oh my goodness, stakeholders love it!
“The next step involved putting users in a room and watching them use Obox. It was one of the most eye opening experiences of our professional careers.
Watching a layman use your product will blow your mind. You cannot even begin to imagine how your users interact with it.
Obox blogged about their usability testing experience
David Perel CEO of Obox
§ Get a team mate who likes talking to people.
§ Get a target user. § Ask the user to do the 3
things the software is for. § Record it.
Your MVUT
Flickr: Lali Masriera/visualpanic
