Download - Validating Ideas with Users
USER-CENTRIC PRODUCT DEVELOPMENT
VALIDATING IDEAS WITH USERS
Sheen YapDreamstake Workshop 6:30pm 9 April 2014 London UK
!..HOW TO CREATE PRODUCTS THAT PEOPLE WILL USE
INTRODUCTION
• Sheen Yap
• 17-years’ experience across web, mobile, iTV
• UX expert, design leader and lean evangelist
• helping startups build better products
WORKSHOP OBJECTIVES
• Early stage startups - new idea, pivoting or existing
• Go through process of validating ideas
• Walk through and apply at each step
• Understand and willing to perform user testing
• Have fun, make mistakes, learn!
HOW DOES THIS FIT IN?
LEAN STARTUP CYCLEEarly stage application!
CUSTOMER DEVELOPMENT
LEAN RUNNING DEV MODEL
HORROR STORYTIME
PROBLEM-SOLUTION FIT - NOW
• Product idea is about problem-solution fit
• Need to solve Real Problems for Real People
• Avoid eventual failure
• Avoid waste in time, effort and money
• Avoid pain in dismantling
• Missed opportunity to pivot to a better idea
..HOW TO CREATE PRODUCTS THAT PEOPLE WILL USE HOW TO AVOID CREATING PRODUCTS NO ONE WILL USE
TEST WITH USERS; TEST EARLY.
HOW DO WE DO IT?
1. Create Model
2. List Assumptions
3. Generate Hypothesis
4. Design Experiment
6. Run experiment
7. Analyse Results
8. Decide Next Steps
5. Experiment (interview)
Captured Data
1. Create Model
2. List Assumptions
3. Generate Hypothesis
4. Design Experiment
6. Run experiment
7. Analyse Results
8. Decide Next Steps
5. Experiment (interview)
Captured Data
1. Create Model
2. List Assumptions
3. Generate Hypothesis
4. Design Experiment
6. Run experiment
7. Analyse Results
8. Decide Next Steps
5. Experiment (interview)
Captured Data
CREATE!MODEL
“IDEAS”
• Ideas normally come as a “solution” i.e. A Thing that has A, B and C Features.
• We are seduced by its perceived tangibility
• No way for anyone else to evaluate its true value to the world
• Can’t tell if it fulfils a genuine need
• Restricts the flexibility for a better solution
MODELLING
INITIAL FOCUS
EXAMPLE - FACEBOOK 2004
Online presence lacking
Hard to stay in contact
College Students
Harvard Students
Personal Profile
Friend Connect
EXERCISE
EXERCISE
• Who are your customer segments?
• What are their problems/pain points/issues?
• Who are your customers?
• What are their problems/pain points/issues?
• Who could be your early adopter?
1. Create Model
2. List Assumptions
3. Generate Hypothesis
4. Design Experiment
6. Run experiment
7. Analyse Results
8. Decide Next Steps
5. Experiment (interview)
Captured Data
LIST!ASSUMPTIONS
ASSUMPTIONS
• “Theory behind the statements”
• Unvalidated, each statement is a figment of our imagination: just because we believe it, does not mean it’s true
• Everything on the canvas are assumptions - we need to methodically test each of them
ASSUMPTION STATEMENT
• We believe that: [user segment] want to do [expected behaviour] because [pain point].
• We believe this [user segment] exist and we can find them at the following [channels]
• We believe that [user segment] will pay [amount] to use our service.
• We believe that [our UVP] will solve the [user segment]’s [problem].
EXAMPLE - FB’04
• Online presence lacking:We believe that lots of students find their current online presence unsatisfactory
• Hard to stay in contact: We believe that many students worry about losing contact with others
EXERCISE
• Pick one of your problems
• Rewrite into an assumption: We believe that [user segment] has this [pain point/issue/need].
1. Create Model
2. List Assumptions
3. Generate Hypothesis
4. Design Experiment
6. Run experiment
7. Analyse Results
8. Decide Next Steps
5. Experiment (interview)
Captured Data
GENERATE!HYPOTHESIS
FALSIFIABLE HYPOTHESIS
• Assumptions need to be testable for validation
• Creating a Falsifiable Hypothesis: i.e. can be proved true or false by empirically testing
• Format: If I were to [perform activity] on [sample subject], then I would be able to [observe an outcome]
EXAMPLE AND EXERCISE
• Assumption: We believe that lots of students find their current online presence unsatisfactory
• Hypothesis: If I were to investigate a number of Harvard students, I would find that 90% of them would be unhappy with their online presence
• Create a hypothesis for your assumption:If I were to [perform activity] on [sample subject], then I would be able to [observe an outcome]
1. Create Model
2. List Assumptions
3. Generate Hypothesis
4. Design Experiment
6. Run
experiment (perform
7. Analyse Results
8. Decide Next Steps
5. Experiment (interview)
Captured Data
DESIGN EXPERIMENT
WHY EXPERIMENTS
• Typical statement: “Built something, put it out there, learnt a lot.”
• Question: What did you REALLY learn?
• Stuff that could be semi-relevant, might be useful later. Erm, hopefully.
• Wasted cycle: still no progress on your most critical problems; worse, could be distracted onto a different direction
AN EXPERIMENT
• Has a learning goal
• Explicit stated hypothesis
• Anticipates observable outcome
• Repeatable
• Avoid biases (try our best to!)
EXPERIMENT TEMPLATE
• Goal: to validate your stated assumption
• Hypothesis: state the falsifiable hypothesis
• Procedure: more on that next…
• Expected Results:
• Actual Result:
EXPERIMENT TEMPLATE
Goal: To validate your stated assumption
Hypothesis: Put your hypothesis here
Procedure: will talk about this next…
Expected Results: come back to this later…
Actual Results: input results after running the experiment…
1. Create Model
2. List Assumptions
3. Generate Hypothesis
4. Design Experiment
6. Run experiment
7. Analyse Results
8. Decide Next Steps
5. Experiment (interview)
Captured Data
EXPERIMENT
WHICH PROCEDURE?
Usability Testing
Ethno-graphy
InterviewsPartici-patory Design
Focus Groups
Eye Tracking
Compe-titive
Research
Diary Study
Bench-marking
Forum Mining
Blogpost Test
Landing Page Tests
PPC Tests
A/B Tests
Customer Feedback
Card Sorting
Wizard of Oz
Neuro Imaging
Data Mining
Data Analysis
Ad Hoc Question-
naire
Intercept Surveys
Email Surveys
Product Stubs
INTERVIEW OBJECTIVES
1. Validate ProblemDo customers have the problem you are solving?
2. Capture Existing Alternatives How do customers solve these problems today?
3. Validate Customer Segments Are these customers the ones with the pain?
INTERVIEWING TIPS
• Set context firstStatement or closed question.
• Then ask open questions Use What/How/Why?
• Follow up questions Why? Tell me more?
• Let them talk; expect messy outcome
INTERVIEW QUESTIONS
• Hypothesis: If I were to investigate a number of Harvard students, I would find that 90% of them would be unhappy with their online presence
• Context setting: I’m trying to find out more about people’s online presence at the moment. Can you tell me if you have an online presence?
• Open questions: (If yes) What do you have? How are you finding it? (If no) Why is that?
• Now write your own question(s) for your hypothesis
EXPERIMENT TEMPLATE
Goal: To validate your stated assumption
Hypothesis: Put your hypothesis here
Procedure: Your interview question(s) to applied to ? people
Expected Results: State your expected outcome
Actual Results: input results after running the experiment…
TEMPLATE EXAMPLE
Goal: To validate our belief that lots of students find their current online presence unsatisfactory
Hypothesis: If I were to investigate a number of Harvard students, I would find that 90% of them would be unhappy with their online presence
Procedure: Interview question(s) to applied to 5 people
Expected Results: At least 4 people will register some complaint about their current online presence
Actual Results:
TEMPLATE EXERCISE
Goal: To validate your stated assumption
Hypothesis: Put your hypothesis here
Procedure: Your interview question(s) to applied to ? people
Expected Results: State your expected outcome
Actual Results:
1. Create Model
2. List Assumptions
3. Generate Hypothesis
4. Design Experiment
6. Run experiment
7. Analyse Results
8. Decide Next Steps
5. Experiment (interview)
Captured Data
RUN EXPERIMENT
INTERVIEW PROCESS
!
• Perform interview with the person next to you
• Interviewee pretend to be target audience
• Try noting/recording in verbatim
• Take turns, 5 minutes each
INTERVIEW EXAMPLE
• Interviewing Harvard Student, circa 2004
• I’m trying to find out more about people’s online presence at the moment. Can you tell me if you have an online presence? “Yes I do.”
• What do you have? How are you finding it?“I have the college directory profile but it’s very limiting in terms of the information it shows, doesn’t tell people much about the real me. I also have a myspace page but it’s kinda kept between my friends and not shown to others in college as stuff there isn’t appropriate. It’s fun but I am spending far too much time maintaining it.”
INTERVIEW EXERCISE
!
• Perform interview with the person next to you
• Interviewee to pretend to be target audience
• Try noting/recording in verbatim
• Take turns, 5 minutes each
1. Create Model
2. List Assumptions
3. Generate Hypothesis
4. Design Experiment
6. Run
experiment (perform
7. Analyse Results
8. Decide Next Steps
5. Experiment (interview)
Captured Data
ANALYSE RESULTS
ANALYSIS GUIDELINES
• Complete ALL interviews first
• Based on what you have recorded:
1. Has it met your expected results, not met them or inconclusive? What does it mean to your hypothesis?
2. What new information have you heard?
3. Any new ideas/insights?
ANALYSIS EXAMPLE
• I have the college directory profile but it’s very limiting in terms of the information it shows, doesn’t tell people much about the real me. I also have a myspace page but it’s kinda kept between my friends and not shown to others in college as stuff there isn’t appropriate. It’s fun but I am spending far too much time maintaining it.
Result: complains registered Hypothesis: validated New info:
college directory limiting
New info: myspace is inappropriate for everyone
New info: Both don't present the “real me”
Insight/Idea: Showing Real Identify is important
ANALYSIS EXERCISE
• Based on what you have recorded:
1. Has it met your expected results, not met them or inconclusive? What does it mean to your hypothesis?
2. What new information have you heard?
3. Any new ideas/insights?
1. Create Model
2. List Assumptions
3. Generate Hypothesis
4. Design Experiment
6. Run
experiment (perform
7. Analyse Results
8. Decide Next Steps
5. Experiment (interview)
Captured Data
DECIDE NEXT STEPS
DECIDING NEXT STEPS
Revise Canvas/Assump-
tionsHypothesis Validated?
New Insight
Gained?
New Insight
Gained?Pick Next
Experiment
Modify Experiment
Experiment Completion
Start New Experiment
Yes
No
InconclusiveBrainstorm New Ideas/
Insights
No
Yes
Yes
No
PROCESS RECAP
1. Create Model
2. List Assumptions
3. Generate Hypothesis
4. Design Experiment
6. Run experiment
7. Analyse Results
8. Decide Next Steps
5. Experiment (interview)
Captured Data
THIS IS LEAN UX
TEST ASSUMPTIONS
✔ ✗
✔
✗
✔
✗ ✗
✗ ✔
✗
✔ ✔
✗ ✔
✔ ✗
✗✔ ✔ ✗ ✔
✔✗
✔
PICK THE RIGHT TOOLS
Usability Testing
Ethno-graphy
InterviewsPartici-patory Design
Focus Groups
Eye Tracking
Compe-titive
Research
Diary Study
Bench-marking
Forum Mining
Blogpost Test
Landing Page Tests
PPC Tests
A/B Tests
Customer Feedback
Card Sorting
Wizard of Oz
Neuro Imaging
Data Mining
Data Analysis
Ad Hoc Question-
naire
Intercept Surveys
Email Surveys
Product Stubs
IT IS ABOUT PEOPLE…
…try walking in their shoes!
CHANGE WILL GET HARDER
MAKING THE NECESSARY CALLS
REFERENCES
QUESTIONS?
Sheen Yap twitter: @sheen uk.linkedin.com/in/sheenyap