building the right product using experiment-driven development · building the right product using...
TRANSCRIPT
![Page 1: Building the Right Product Using Experiment-Driven Development · Building the Right Product Using Experiment-Driven Development Mike Hall | Monday 2:30 PM | Room 200 E Experiment-Driven](https://reader034.vdocument.in/reader034/viewer/2022042304/5ecfb73f1135c57f253bdc4d/html5/thumbnails/1.jpg)
Building the Right Product Using Experiment-Driven Development
!1
![Page 2: Building the Right Product Using Experiment-Driven Development · Building the Right Product Using Experiment-Driven Development Mike Hall | Monday 2:30 PM | Room 200 E Experiment-Driven](https://reader034.vdocument.in/reader034/viewer/2022042304/5ecfb73f1135c57f253bdc4d/html5/thumbnails/2.jpg)
Mike HallSenior Agile Coach/Trainer @MikeH_agile [email protected] www.agilevelocity.com
Accelerate Agility Transformation | Training | Coaching
2
![Page 3: Building the Right Product Using Experiment-Driven Development · Building the Right Product Using Experiment-Driven Development Mike Hall | Monday 2:30 PM | Room 200 E Experiment-Driven](https://reader034.vdocument.in/reader034/viewer/2022042304/5ecfb73f1135c57f253bdc4d/html5/thumbnails/3.jpg)
Predict AccelerateLearnAlign Adapt
Status Quo Chaos & ResistanceIn
tegr
atio
n &
Prac
tice
New Status Quo
The goal for the transformation cannot be to do Agile. Understanding and communicating the business objectives that will be achieved with the transformation is a critical first step.
Through Agile training and coaching, teams and leadership are equipped with new techniques and an understanding of how Agile works.
Ownership of processes are transferred to an empowered team and a culture of continuous improvement is put in place.
Teams harden these newly learned practices and become more disciplined in order to deliver working product in a predictable and iterative manner.
Once the teams become disciplined and predictable, we can focus on team and organizational improvements to optimize across the full delivery cycle and shorten time to market.
Agile will begin to permeate throughout the organization and executive leadership, enabling empowered teams and adaptive leadership to respond to ever-changing market demands as they have transformed to an organization with true Agility.
@agile_velocity
!3
![Page 4: Building the Right Product Using Experiment-Driven Development · Building the Right Product Using Experiment-Driven Development Mike Hall | Monday 2:30 PM | Room 200 E Experiment-Driven](https://reader034.vdocument.in/reader034/viewer/2022042304/5ecfb73f1135c57f253bdc4d/html5/thumbnails/4.jpg)
What if we found ourselves building something that nobody wanted?
In that case, what did it matter if we did it on time, on budget, and with high quality?
!4
Has this ever happened to you?
![Page 5: Building the Right Product Using Experiment-Driven Development · Building the Right Product Using Experiment-Driven Development Mike Hall | Monday 2:30 PM | Room 200 E Experiment-Driven](https://reader034.vdocument.in/reader034/viewer/2022042304/5ecfb73f1135c57f253bdc4d/html5/thumbnails/5.jpg)
Let’s celebrate our failures! ???
!5
$200M SW/HW development investment, 12 months.
Result: $0 revenues.
![Page 6: Building the Right Product Using Experiment-Driven Development · Building the Right Product Using Experiment-Driven Development Mike Hall | Monday 2:30 PM | Room 200 E Experiment-Driven](https://reader034.vdocument.in/reader034/viewer/2022042304/5ecfb73f1135c57f253bdc4d/html5/thumbnails/6.jpg)
Experiment-Driven Development (EDD)!6
![Page 7: Building the Right Product Using Experiment-Driven Development · Building the Right Product Using Experiment-Driven Development Mike Hall | Monday 2:30 PM | Room 200 E Experiment-Driven](https://reader034.vdocument.in/reader034/viewer/2022042304/5ecfb73f1135c57f253bdc4d/html5/thumbnails/7.jpg)
How long is theRelease cycle?
(From Concept to Customer)
How many features are released
together?
How is success determined?
Your Product TodayYour Reality
3
!7
![Page 8: Building the Right Product Using Experiment-Driven Development · Building the Right Product Using Experiment-Driven Development Mike Hall | Monday 2:30 PM | Room 200 E Experiment-Driven](https://reader034.vdocument.in/reader034/viewer/2022042304/5ecfb73f1135c57f253bdc4d/html5/thumbnails/8.jpg)
!8
Nordstrom Video
![Page 9: Building the Right Product Using Experiment-Driven Development · Building the Right Product Using Experiment-Driven Development Mike Hall | Monday 2:30 PM | Room 200 E Experiment-Driven](https://reader034.vdocument.in/reader034/viewer/2022042304/5ecfb73f1135c57f253bdc4d/html5/thumbnails/9.jpg)
Debrief Video Impact?
How long is theRelease cycle?
(From Concept to Customer)
How many features are released
together?
How is success determined?
Your Product Today
3
!9
![Page 10: Building the Right Product Using Experiment-Driven Development · Building the Right Product Using Experiment-Driven Development Mike Hall | Monday 2:30 PM | Room 200 E Experiment-Driven](https://reader034.vdocument.in/reader034/viewer/2022042304/5ecfb73f1135c57f253bdc4d/html5/thumbnails/10.jpg)
Activity:
Huddle up in pairs or 3s.
Discuss what was different in this approach vs. your current project/product.
!10
3
![Page 11: Building the Right Product Using Experiment-Driven Development · Building the Right Product Using Experiment-Driven Development Mike Hall | Monday 2:30 PM | Room 200 E Experiment-Driven](https://reader034.vdocument.in/reader034/viewer/2022042304/5ecfb73f1135c57f253bdc4d/html5/thumbnails/11.jpg)
Debrief - Potential Differences:
!11
“Never work on things that are
not valued by the customer!”
![Page 12: Building the Right Product Using Experiment-Driven Development · Building the Right Product Using Experiment-Driven Development Mike Hall | Monday 2:30 PM | Room 200 E Experiment-Driven](https://reader034.vdocument.in/reader034/viewer/2022042304/5ecfb73f1135c57f253bdc4d/html5/thumbnails/12.jpg)
Develop a short list of product hypotheses:
• Consider “baked-in” assumptions
• Define testable hypotheses from assumptions
• Prioritize these hypotheses - based on potential learnings
EDD Step 1: Start With a Product Hypotheses
!12
Types of hypotheses:
• Customer
• Problem
• Solution
• Value
• Growth
![Page 13: Building the Right Product Using Experiment-Driven Development · Building the Right Product Using Experiment-Driven Development Mike Hall | Monday 2:30 PM | Room 200 E Experiment-Driven](https://reader034.vdocument.in/reader034/viewer/2022042304/5ecfb73f1135c57f253bdc4d/html5/thumbnails/13.jpg)
Product Hypothesis - Examples
!13
Growth hypothesis: “An in-store available iPad app will increase sunglass sales by 20%.”
Problem hypothesis: “Potential customers have no obvious way to compare different versions of sunglasses other than sequential looks in a mirror.”
Value hypothesis: “Allowing the customer to ‘compare’ themselves side-by-side in different sunglasses will enhance their overall satisfaction at our sunglass station by 40%.”
Solution hypothesis: “When offered, more than 50% of potential customers will choose to use an iPad app to make it easier to try on sunglasses.”
![Page 14: Building the Right Product Using Experiment-Driven Development · Building the Right Product Using Experiment-Driven Development Mike Hall | Monday 2:30 PM | Room 200 E Experiment-Driven](https://reader034.vdocument.in/reader034/viewer/2022042304/5ecfb73f1135c57f253bdc4d/html5/thumbnails/14.jpg)
Activity:
Using an index card, write out a testable hypothesis for a product you are currently working on (or recent one).
• Hint: consider your baked-in assumptions!
Identify type as customer, problem, solution, value, or growth type.
!14
3 3
Share with a new friend from another table you have not met yet!
![Page 15: Building the Right Product Using Experiment-Driven Development · Building the Right Product Using Experiment-Driven Development Mike Hall | Monday 2:30 PM | Room 200 E Experiment-Driven](https://reader034.vdocument.in/reader034/viewer/2022042304/5ecfb73f1135c57f253bdc4d/html5/thumbnails/15.jpg)
EDD Step 2: Identify Experiments
!15
Take your highest priority Hypothesis and define the smallest experiment that will prove or disprove it.
Examples: • Hypothesis: An in-store available iPad app will increase sunglass sales by 20%.
Experiment 1: Develop a basic iPad app that allows customer to compare what they look like with various sunglasses.
Experiment 2: Roll out to 10 stores only, measure sales impact.
• Hypothesis: Side-by-Side is a better comparison technique than multi-pic.
Experiment 1: Create a multi-layout technique showing up to 6 different pics in the main view
Experiment 2: Demo both approaches (side-by-side and multi-view) to at least 100 customers. Measure preference %.
![Page 16: Building the Right Product Using Experiment-Driven Development · Building the Right Product Using Experiment-Driven Development Mike Hall | Monday 2:30 PM | Room 200 E Experiment-Driven](https://reader034.vdocument.in/reader034/viewer/2022042304/5ecfb73f1135c57f253bdc4d/html5/thumbnails/16.jpg)
Activity: On the back of your index card, write 2 - 3 experiments for your product hypothesis.
!16
3 3
Share your product hypothesis and experiments with a different new friend.
![Page 17: Building the Right Product Using Experiment-Driven Development · Building the Right Product Using Experiment-Driven Development Mike Hall | Monday 2:30 PM | Room 200 E Experiment-Driven](https://reader034.vdocument.in/reader034/viewer/2022042304/5ecfb73f1135c57f253bdc4d/html5/thumbnails/17.jpg)
Experiment Backlogs
• Similar to Scrum product backlog
• But is learning-based prioritization
!17
• List of all experiments 1..n
Work items, mocks, research, etc.
• Tagged with Hypothesis description
![Page 18: Building the Right Product Using Experiment-Driven Development · Building the Right Product Using Experiment-Driven Development Mike Hall | Monday 2:30 PM | Room 200 E Experiment-Driven](https://reader034.vdocument.in/reader034/viewer/2022042304/5ecfb73f1135c57f253bdc4d/html5/thumbnails/18.jpg)
Nielsen DCAPI project - Learning Map
MVP 1
Hypotheses
Experiments
!18
MVP 2
![Page 19: Building the Right Product Using Experiment-Driven Development · Building the Right Product Using Experiment-Driven Development Mike Hall | Monday 2:30 PM | Room 200 E Experiment-Driven](https://reader034.vdocument.in/reader034/viewer/2022042304/5ecfb73f1135c57f253bdc4d/html5/thumbnails/19.jpg)
Build
Experiments Measure
Learn
MVPe
19
EDD Step 3: Run Experiments
![Page 20: Building the Right Product Using Experiment-Driven Development · Building the Right Product Using Experiment-Driven Development Mike Hall | Monday 2:30 PM | Room 200 E Experiment-Driven](https://reader034.vdocument.in/reader034/viewer/2022042304/5ecfb73f1135c57f253bdc4d/html5/thumbnails/20.jpg)
Experiments - Validated Learnings
• Build out the MVP • Measure progress based on validated learning from end user! • Use modified storyboard showing Validated column
To Do In Work Done Validated
EXP-1
EXP-2
EXP-3
EXP-4
EXP-5
!20
![Page 21: Building the Right Product Using Experiment-Driven Development · Building the Right Product Using Experiment-Driven Development Mike Hall | Monday 2:30 PM | Room 200 E Experiment-Driven](https://reader034.vdocument.in/reader034/viewer/2022042304/5ecfb73f1135c57f253bdc4d/html5/thumbnails/21.jpg)
Experiment Test Iteration (ETI)Scrum: fixed iteration length
ETI: variable iteration length
ETI 1 3 days
ETI 2 5 days
ETI 3 9 days
ETI 4 17 days
ETI 5 6 days
ETI 6 7 days
MVP1 MVP2 MVP3 MVP4 MVP5 MVP6
1 2 3 4 1 2 3 4 1 2 3 4 1 2 3 4 1 2 3 4 1 2 3 4
Sprint 1 Weeks
Sprint 2 Weeks
Sprint 3 Weeks
Sprint 4 Weeks
Sprint 5 Weeks
Sprint n Weeks
!21
![Page 22: Building the Right Product Using Experiment-Driven Development · Building the Right Product Using Experiment-Driven Development Mike Hall | Monday 2:30 PM | Room 200 E Experiment-Driven](https://reader034.vdocument.in/reader034/viewer/2022042304/5ecfb73f1135c57f253bdc4d/html5/thumbnails/22.jpg)
ETI Review• Dev team demos their working
software
• Team discusses validated
learnings with stakeholders
• Decision: Pivot/Persevere/Quit
!22
• Plus:
- How is the team feeling about the
baked-in assumptions?
- Are there any assumptions not
previously considered?
![Page 23: Building the Right Product Using Experiment-Driven Development · Building the Right Product Using Experiment-Driven Development Mike Hall | Monday 2:30 PM | Room 200 E Experiment-Driven](https://reader034.vdocument.in/reader034/viewer/2022042304/5ecfb73f1135c57f253bdc4d/html5/thumbnails/23.jpg)
SIMULATION!23
![Page 24: Building the Right Product Using Experiment-Driven Development · Building the Right Product Using Experiment-Driven Development Mike Hall | Monday 2:30 PM | Room 200 E Experiment-Driven](https://reader034.vdocument.in/reader034/viewer/2022042304/5ecfb73f1135c57f253bdc4d/html5/thumbnails/24.jpg)
Hypothesis:
“A tower made of spaghetti, string, tape, and a single marshmallow on top can be made to stand over 28 inches tall.”
20 Sticks of Spaghetti
1 Yard of Tape
1 Yard of String 1 Marshmallow
Supplies
• Tower must be free-standing (no support other than floor/table base)• Tower must remain standing indefinitely • Marshmallow must be on top and remain intact• 18 minutes total - run multiple BML loops• Highest measurement from base to top wins!
Rules
!24
![Page 25: Building the Right Product Using Experiment-Driven Development · Building the Right Product Using Experiment-Driven Development Mike Hall | Monday 2:30 PM | Room 200 E Experiment-Driven](https://reader034.vdocument.in/reader034/viewer/2022042304/5ecfb73f1135c57f253bdc4d/html5/thumbnails/25.jpg)
Activity:
Using post-it notes, identify 3 - 5 Experiments to try, 1 per post-it note.
Order the Experiment cards based on potential learnings.
!25
5
18-minute timeframe begins when instructor says “Ready/Set/Go”.
Build/Measure/Learn loops: adjust the experiments as you go, pivot/persevere.
![Page 26: Building the Right Product Using Experiment-Driven Development · Building the Right Product Using Experiment-Driven Development Mike Hall | Monday 2:30 PM | Room 200 E Experiment-Driven](https://reader034.vdocument.in/reader034/viewer/2022042304/5ecfb73f1135c57f253bdc4d/html5/thumbnails/26.jpg)
Debrief
!26
Observations?
How did you use your validated learnings?
Did your Experiments Backlog change as you executed the iterations?
Would more planning time have really helped?
Did any team PIVOT to a new hypothesis (different approach, different structure type)?
![Page 27: Building the Right Product Using Experiment-Driven Development · Building the Right Product Using Experiment-Driven Development Mike Hall | Monday 2:30 PM | Room 200 E Experiment-Driven](https://reader034.vdocument.in/reader034/viewer/2022042304/5ecfb73f1135c57f253bdc4d/html5/thumbnails/27.jpg)
Ted Talk - Marshmallow Challenge
!27
![Page 28: Building the Right Product Using Experiment-Driven Development · Building the Right Product Using Experiment-Driven Development Mike Hall | Monday 2:30 PM | Room 200 E Experiment-Driven](https://reader034.vdocument.in/reader034/viewer/2022042304/5ecfb73f1135c57f253bdc4d/html5/thumbnails/28.jpg)
!28
WRAP
![Page 29: Building the Right Product Using Experiment-Driven Development · Building the Right Product Using Experiment-Driven Development Mike Hall | Monday 2:30 PM | Room 200 E Experiment-Driven](https://reader034.vdocument.in/reader034/viewer/2022042304/5ecfb73f1135c57f253bdc4d/html5/thumbnails/29.jpg)
!29
EDD in a Nutshell
Product Hypotheses
Experiments
Build-Measure-Learn Loops
The Right Product!
![Page 30: Building the Right Product Using Experiment-Driven Development · Building the Right Product Using Experiment-Driven Development Mike Hall | Monday 2:30 PM | Room 200 E Experiment-Driven](https://reader034.vdocument.in/reader034/viewer/2022042304/5ecfb73f1135c57f253bdc4d/html5/thumbnails/30.jpg)
We value • Validated learning over seemingly-reasonable
assumptions
• Data-driven decisions over plausible-sounding arguments
• Building minimum learning products over additional features
• The courage to build the right thing over something that works
A Post-Agile Manifesto!30
![Page 31: Building the Right Product Using Experiment-Driven Development · Building the Right Product Using Experiment-Driven Development Mike Hall | Monday 2:30 PM | Room 200 E Experiment-Driven](https://reader034.vdocument.in/reader034/viewer/2022042304/5ecfb73f1135c57f253bdc4d/html5/thumbnails/31.jpg)
Call to Action!!31
![Page 32: Building the Right Product Using Experiment-Driven Development · Building the Right Product Using Experiment-Driven Development Mike Hall | Monday 2:30 PM | Room 200 E Experiment-Driven](https://reader034.vdocument.in/reader034/viewer/2022042304/5ecfb73f1135c57f253bdc4d/html5/thumbnails/32.jpg)
AGILE VELOCITY
David Hawks
Mike Hall
Reese Schmit
Don’t Miss Other Agile Velocity Speakers
Building the Right Product Using Experiment-Driven DevelopmentMike Hall | Monday 2:30 PM | Room 200 EExperiment-Driven Development (EDD) is an emerging Agile approach designed to ensure that we always build the right product. Join us to explore how to create a culture of experimentation within your team
10 Steps to a Successful Enterprise Agile Transformation: The Precipitous Path to Predictability!Mike Hall | Wednesday 11:00 AM | Room 200 ENavigate the precipitous path to predictability through 10 tangible steps to a successful enterprise Agile Transformation.
Why Are We Stuck? Getting back To Continuous Improvement Reese Schmit | Monday 1:00 PM | Room 200 GHWhen things are going well, its hard to find motivation to improve. In this session, you will learn the symptoms of a stuck team and discover tools to get them back on the path of relentless improvement.
The Post Project Era: The Future of Agile David Hawks | Monday 11:00 AM | Room 200 EIt’s time to shift from a project-driven mindset to a value-driven mindset. David explores how traditional practices slow product development and what needs to change to support this shift.
Move Beyond User Stories… What’s Next?David Hawks | Wednesday 9:15 AM | 101 EThe current epic and user story process is stale and needs to go. In this hands-on workshop, you’ll learn a new discovery-centered model driven by objectives, hypotheses, and experiments.
![Page 33: Building the Right Product Using Experiment-Driven Development · Building the Right Product Using Experiment-Driven Development Mike Hall | Monday 2:30 PM | Room 200 E Experiment-Driven](https://reader034.vdocument.in/reader034/viewer/2022042304/5ecfb73f1135c57f253bdc4d/html5/thumbnails/33.jpg)
Give us a business card or your name and email on an index card, and we will send you the following: •The slides presented today
•A video on a related topic
•An article on Next Level Agile
Would you like More?
!33