lean startup: experiment-driven product developmentfiles.meetup.com/1508927/dfw scrum - lean...

Lean Startup: Experiment-Driven

Product Development

Mike HallImprovingmike.hall@improving.com

972.757.9027

2

Money Is No Object … ?

Operations Platform for Data Communication System (DCS)

CT2-Plus wireless systems

CEMAPS – cellular performance system

Lean Startup

3

4

Applicability

It’s for everyone: startups, new product development, new features, etc.

5

Mantra

The goal of any project is to figure out the right thing to build.

“What if we found ourselves building something that nobody wanted? In that case, what did it matter if we

did it on time and on budget?”

Look carefully at this picture – what is wrong?

6

Principles

1. Start with your project/feature assumptions

• Reword these as hypotheses

• Continuously!

Begin with a clear hypothesis that makes

predictions about what is supposed to happen.

7

Principles

2. Build-Measure-Learn: Short dev cycles of “validated learning”

• Build experiments that will test your assumption hypotheses

- Experiments are your “stories”

- Prioritize based on … ?

• Measure results

• Learn – what do the results tell us?

“The measure of an effective team is how

much validated learning did we achieve

(as opposed to how much did we build).”

8

Principles

3. Adjust project direction based on BML validated learnings

• Pivot or Persevere (or Quit)

“There is no bigger destroyer of creative potential than themisguided decision to persevere.”

PIVOTPERSEVERE

QUIT

9

Principles (others)

4. Small batch size

5. Minimum viable (learning) product

10

Principles (others)

6. Engine of growth

7. Adaptive organization

8. Innovation accounting

Application toAgile Projects

11

12

DCAPI Vision Board

Target Group Needs Product Value

For clients who have a need for capturing census-based usage analytics on their connected devices/applications, Data Collection API (DCAPI) is a cloud-based service that provides a simple easy-to-understand way of reporting measurements.

Unlike classic embedded SDK approaches, the DCAPI will provide a direct reporting experience based on web service calls.

Digital customers• CBS Interactive• MobiTV• A&E Apps• Crown Media• Univision Apps• Pandora• Viacom• Fox News• DirecTV• NBCU Apps• AT&T• JW Player• Yelp• Roku• Xbox• Connected TVs• PlayStation

• Ease of measurement reporting

• Use of familiar programmatic

approach

• Less software development

• No need to download/integrate

SDK

• One solution for all digital

• Cloud-based

• Transparent evolution

• Linear scaling as demand

grows

• Fault tolerant

• Increase revenues

• Satisfy pent-up

demand

• Increase digital

footprint

• 1-stop-shop

Assumptions Hypotheses• Customers will prefer DCAPI over the

embedded SDK

• DCAPI will make it easier to certify apps

• DCAPI can handle a large amount of users

• DCAPI will need a super-fast DB

• An early release to friendly customers will

provide good feedback

• > 80% of all customers will prefer DCAPI

• DCAPI can be self-certified by customers

• DCAPI can handle 50K simultaneous

sessions

• Redis is the best DB for DCAPI

• An initial release can be built with limited (but

valuable) functionality for early adopters

Vision Statement

13

DCAPI Learning Map

Hypotheses

Experiments:Stories, Work Items

14

Experiment Backlogs

• Similar to Scrum product backlog

• But is learning-based prioritization

• List of all experiments 1..n

- Stories, work items, research, etc.

• Tagged with Hypothesis name/description

15

Minimum Learning Product (MLP)

• Similar to MVP, but typically smaller

• Learning-based more than product-based

• Choosing the MLP replaces classic Scrum sprint planning

MLP MLP MLP

16

DCAPI Learning Map

MLP 1

17

Experiment Test Iteration (ETI)

Scrum: fixed iteration length

ETI: variable iteration length

ETI 13 days

ETI 25 days

ETI 39 days

ETI 417 days

ETI 56 days

ETI 67 days

MLP MLP MLP MLP MLP MLP

18

ETI

• Build out the MLP

• Measure progress based on validated learning

• Use modified storyboard showing Validated column

Story To Do In Work Done Validated

19

ETI Review

• Dev team demos their working software

• Team discusses validated learnings with stakeholders

• Decision: pivot, persevere, or quit

20

ETI Retrospective

• Team discusses- What went well, what did not go well

- How to get better

• A spirit of “continuous improvement”

• Plus:- How is the team feeling about the assumptions?

- Are there any assumptions not identified previously?

21

Rinse & Repeat

• Build series of MLPs to reach final launchable product

• Use innovation accounting to “tune the growth engine”

• Be courageous in pivot/persevere/quit decisions

ProjectSimulation

22

23

Project Simulation

• Goal: build the highest tower possible!

• Supplies- Dowels – 6 thick, 6 thin (do not break!)

- Spaghetti sticks – 10 (you can break)

- Marshmallows – 6

- Tape – 3 feet

- String – 3 feet

- Scissors

- 20 Index cards – for capturing experiments only

24

Rules

• Teams of 4 – 5

• Everyone must participate (no observers)

• Team must use Lean Startup BML loops

• 30 minutes total time

• Judging the winner:- A marshmallow must be on the top!

- Marshmallows must remain intact

- Tower must be free standing - no means of

support other than the table or floor

- Highest measurement from tower base to

top of marshmallow

25

Guidance

• Together: Round 1 Prep – 5 minutes- Identify your hypotheses

- Write your experiments down on index cards

- Prioritize your experiments

- Identify which experiments you will do in iteration 1

• When I signal, begin your BML loop

• At end of each BML loop, keep going- Team decision: Pivot or Persevere (no quitting!)

- Team decision: experiments for next BML loop

- Update your cards for next iteration experiments

- Start next BML loop

• Rinse & Repeat – until time expires

26

The Winner Is …

27

Debrief

• Observations?

• Did discussing your

assumptions/hypotheses help?

• What were your experiments?

• Did you ever pivot?

• Did BML learnings change your

subsequent experiments?

• Do you hate marshmallows now?

Get Started Now!

28

29

Conclusion

• Lean Startup principles can and should be used in Agile projects

- To help insure we build the right thing

• This could be the next major evolution of Agile!

“If we stopped wasting people’s time, what would they do with it?

We have no real concept of what is possible.”

30

Team Dynamics

• Think of your Scrum team as a small “innovation factory”- Responsible for code and/or artifacts that prove/disprove a hypothesis

- Continuous innovation

• Practicing the art of “genchi genbutsu”- “Go and see”

- The only way to truly understand the requirements is to get out of the

office and spend time with the customer

- Gemba – the real place

- Don’t rely on information from other sources

31

Gemba Walk

• Go see the actual process- Purposeful attempt to learn what is really going on- Direct customer interaction- Ask questions- Show respect- Learn

32

Team Dynamics (cont)

• ScrumMaster becomes “shusa”- Chief engineer responsible for guiding the product to success

- Guides team on experiments to MLPs to product

33

Introducing: Gemba

Gemba: a validated learning Agile method

Gemba

Scrum Lean Startup Lean

34

Gemba Manifesto

We value• Validated learning over seemingly-reasonable assumptions

• Data-driven decisions over plausible-sounding arguments

• Building minimum learning products over additional features

• The courage to build the right thing over something that works

35

Final Q&A

THE END

Mike HallImprovingmike.hall@improving.com

972.757.9027