taking a hypothesis driven approach
TRANSCRIPT
2
Assumptions
Assumptions
Assumptions
Assumptions
Assumptions
Assumptions
Assumptions
Assumptions
Assumptions
AssumptionsAssumptions
Assumptions
AssumptionsAssumptions
Assumptions
Assumptions
Assumptions
CREATING A HYPOTHESIS
6
We believe that <this change>
Will result in <this outcome>.
We will know we have succeeded when <we see change in X metric>.
Retweet: “I’ll try a #HypothesisDrivenApproach #AgileAus @nicolaboyle27 @remoterobbo”
SKYPE FOR STAFF
7
We believe that enabling Skype functionality to staff
Will result in less staff travel and quicker turnaround
times on decision making.
We will know we have succeeded when we see that the flights taken by staff, for face to face meetings, have been reduced by 5%.
Retweet: “I’ll try a #HypothesisDrivenApproach #AgileAus @nicolaboyle27 @remoterobbo”
MVP - THE HYPOTHESIS’S BEST FRIEND
8Retweet: “I’ll try a #HypothesisDrivenApproach #AgileAus @nicolaboyle27 @remoterobbo”
METRICS THAT MATTER
9
Acquisition
Activation
Retention
Revenue
Referral
AARRR “PIRATE”FUNNEL METRICS
Retweet: “I’ll try a #HypothesisDrivenApproach #AgileAus @nicolaboyle27 @remoterobbo”
- Dave McClure
Acquisition - Number of people who visit your service
Activation - Number of people who have a good initial response
Retention - Number of people who come back for more
Revenue - Number of people from the cohort who engage in revenue-creating activity Referral - Number of people from the cohort who refers other users
ITERATE TO VALIDATE
10
THE VALIDATED LEARNING LOOP
• REDUCED RISK & UNCERTAINTY
• MORE VALUE SOONER
• LAYER-IN COMPLEXITY
Create hypothesis
Design and run
experiment
Study results
Evolve model and
implement changes
Retweet: “I’ll try a #HypothesisDrivenApproach #AgileAus @nicolaboyle27 @remoterobbo”
CREATE AN INNOVATION CULTURE
12Retweet: “I’ll try a #HypothesisDrivenApproach #AgileAus @nicolaboyle27 @remoterobbo”
DISRUPTS
13Retweet: “I’ll try a #HypothesisDrivenApproach #AgileAus @nicolaboyle27 @remoterobbo”
ACCEPT OR CANCELPHONE, TEXT, EMAIL OR NOTIFICATION
MANAGE BOOKING ONLINE
FROM REQUIREMENTS TO HYPOTHESES
14
We believe allowing passengers to change their flight within a seven day period
Will result in a better customer experience as passengers will not have to call to manage their booking.
We will know we have succeeded when 10% of customers select a new flight within the seven day period.
Create hypothesis
Design and run
experiment
Study results
Evolve model and
implement changes
SEVEN DAY CHANGE
Retweet: “I’ll try a #HypothesisDrivenApproach #AgileAus @nicolaboyle27 @remoterobbo”
BUILD AN MVP
15Retweet: “I’ll try a #HypothesisDrivenApproach #AgileAus @nicolaboyle27 @remoterobbo”
THREE DAY CHANGE
Create hypothesis
Design and run
experiment
Study results
Evolve model and
implement changes
CHECK THE RESULTS
16
• Most passengers accepted new flight.
• Some passengers changed on day or cancelled.
• Very few passengers changed to next day and fewer changed on the third day.
Retweet: “I’ll try a #HypothesisDrivenApproach #AgileAus @nicolaboyle27 @remoterobbo”
Create hypothesis
Design and run
experiment
Study results
Evolve model and
implement changes
CONTINUE, PIVOT, STOP
17
• Cost to build seven day change deemed not worth the ROI.
• Seven day change feature stopped.
• Team able to pick up more valuable feature.
Retweet: “I’ll try a #HypothesisDrivenApproach #AgileAus @nicolaboyle27 @remoterobbo”
Create hypothesis
Design and run
experiment
Study results
Evolve model and
implement changes
FROM DONE TO MONITORING
18
Team walls and portfolio wall now have Monitoring instead of Done.
Definition of done to move out of monitoring column is ‘Measure outcome’
Retweet: “I’ll try a #HypothesisDrivenApproach #AgileAus @nicolaboyle27 @remoterobbo”
19
BECOME A DATA-LED BUSINESS
Retweet: “I’ll try a #HypothesisDrivenApproach #AgileAus @nicolaboyle27 @remoterobbo”
BUILD MEASUREMENT IN
20
“Without data you’re just another person with an opinion” - W. Edwards Deming
-
• Agreed before development.
• An acceptance criteria for every story.
• Part of the way of working - it’s not done until you’ve studied the results and acted on them.
Retweet: “I’ll try a #HypothesisDrivenApproach #AgileAus @nicolaboyle27 @remoterobbo”
ISOLATE CHANGE
21
We believe providing the ability to add an extra bag prior to payment only
Will result in increased customer satisfaction and speed to complete bookings.
We will know we have succeeded when:
• There is reduction in complaints from customers related to overloading of information;
• Decrease in time to complete a booking (from search to payment);
• There is no statistically significant drop-off in customers adding extra bags.
Retweet: “I’ll try a #HypothesisDrivenApproach #AgileAus @nicolaboyle27 @remoterobbo”
ADD EXTRA BAGS
TRANSFORM YOUR ORGANISATION
23Retweet: “I’ll try a #HypothesisDrivenApproach #AgileAus @nicolaboyle27 @remoterobbo”
GUILD WORKSHOP
24Retweet: “I’ll try a #HypothesisDrivenApproach #AgileAus @nicolaboyle27 @remoterobbo”
TRAINING BUSINESS ANALYSTS ON HYPOTHESIS DRIVEN DEVELOPMENT
We believe running a one hour session where we introduce the concept, core theory and have participants
build a hypothesis based on their existing project
Will result in an increase in attendees applying a hypothesis driven approach in their teams.
We will know we have succeeded when the average score for attendees is at least a three on a self-assessment scale.
MEASURING SUCCESS
25Retweet: “I’ll try a #HypothesisDrivenApproach #AgileAus @nicolaboyle27 @remoterobbo”
1. I know nothing about hypothesis driven development.
2. I know a little, but not how to apply it.
3. I could work in a team using it.
4. I could apply the concepts and explain it to others.
5. Move over, I will be presenting this session.
SELF-ASSESSMENT
SECOND TIME RUNNING THE SESSION
PIVOT
26Retweet: “I’ll try a #HypothesisDrivenApproach #AgileAus @nicolaboyle27 @remoterobbo”
We believe running a ninety minute session where we introduce the concept, core theory and have participants
build a hypothesis based on a fictional project
Will result in an increase in attendees applying a hypothesis driven approach in their teams.
We will know we have succeeded when the average score for attendees is at least a three on a self-assessment scale.
RESOURCES
28
The Lean Startup - Eric Ries
Entrepreneur/Author - Barry O’Reilly
The Lean Enterprise - Jez Humble, Joanne
Molesky and Barry O’Reilly
Retweet: “I’ll try a #HypothesisDrivenApproach #AgileAus @nicolaboyle27 @remoterobbo”
THANK YOU
[email protected] [email protected]
“I’ll try a #HypothesisDrivenApproach #AgileAus @nicolaboyle27 @Remoterobbo”