siny leanstartup introduction | reduce waste, run experiments!

MINIMUM VIABLE PRESENTATION Lean Startup

Jonathan Bertfield & Adam Berk, w/ John Lynn, @jmlynn7 @startupinst

2

ELIMINATE WASTE!!!!

3

RUN EXPERIMENTS (like this one)

THANK YOU:)

@adamberk @berters

what will I learn?

1. What is LS?2. Why should you care?3. How do you do it (if you decide to care) 4. Discussion of Topics:

• What is an MVP (what are some great examples?)• What do I test, what do I measure?• What do I do with my learnings?

5. Get INTO and THROUGH BML once! (you are not practicing until you START the second loop)

Jonathan Bertfield

• 20 Years in Product Management

• Consulting clients: Pearson Inc., Disney, Associated Press, News Corp. Sesame Workshop, Business Insider, American Press Institute

• Coach at General Assembly Enterprise, Lean Startup Co Faculty, NYU Stern

Adam Berk

• What problem are you solving? I help teams pick their riskiest assumptions and run experiments.

• Currently at Pearson with >>• Founded a sharing economy site

with the opposite result of AirBnb

we believe that you…1. have heard the term LS before today

(95%)2. have read “the book” (60%)3. believe that LS is “at least worth

considering” (80%)4. have run an experiment or thought about

it in your lifetime (maybe) (60%)5. want (are willing) to run (more)

experiments with your (new/future) team (80% now 95% end of preso)

what is a startup? “A human institution designed to create a new product or service under conditions of extreme uncertainty. “

- Eric Riesauthor, The Lean Startup

A method to systematically address uncertainty through rapid iteration and market learning.

what is the Lean Startup?

we WILL cover the followingbuzzwords. MVP. CUSTDEV. PIVOT...• Entrepreneurs• Startups• Uncertainty (Product / Market / Model)• Minimum Viable Product / Pre MVP• Experiments

Assumptions*Hypotheses*(Vanity) metrics, Customer development

• Validated Learning• (how to do test and measure the right things) • Pivots

15

Defining waste.

Defining waste. • Building things that no one wants • GOING TOO SLOWLY (wasting time) • NOT LEARNING FAST ENOUGH • Doing things the “old way” because “that is

how they are done”• Solutions looking for problems (most of the

time) ((almost ALL the time in early stage

startups))

why there’s waste. • cognitive biases• poor communication • fear of penalty kicks down the

center• sunk cost syndrome• MORE?

19

Don't build this.

21

OR THIS

if you build without testing, you are probably building the wrong thing... 22

23

What problem are you solving?

24

25

NO ONE NEEDS A WASHING MACHINE

THEY NEED CLEAN CLOTHES

26

27

Scientists do not coddle ideas. They crash test them. They run them

into a brick wall at sixty miles per hour. If the

idea is sound the pieces will be of the wall. 28

29

Minimum Viable Product (MVP)

Experiment that helps you validate (or invalidate) hypotheses about the value or growth potential for a new product

An MVP helps you answer a specific question about one of your assumptions (as fast as possible and helps you understand the WHY)

scientific method

1. Identify problems to generate ideas2. Identify and prioritize the underlying

assumptions behind those ideas3. Generate hypotheses and predictions 4. Define and deliver tests that expose

opportunities for learning5. Use resulting data (learnings) to inform

activity6. Repeat forever (remember the second loop

in the roller coaster)

Test the riskiest things

first(next)

32

High Risk

Low Risk

UnknownKnown

components of an example hypothesis

We believe [doing this]

for [these people]

will [achieve this outcome]

We will know this to be true when

we see [this business outcome]

by [this specific timeframe]

components of an example hypothesis

TACTICAL

TESTABLE

components of an example hypothesis

We believe [building a network of independent drivers]

for [New York]

will [make it faster and easier to grab a cab a peak times]

We will know this to be true when

we see [20 drivers with 20 rides/day ]

by [1 month from now]

Devin’s hypothesis

We believe [building a network of independent drivers]

for [New York]

will [make it faster and easier to grab a cab a peak times]

We will know this to be true when

we see [20 drivers with 20 rides/day ]

by [1 month from now]

components of an example hypothesis

We believe [creating content based on journalists passions]

for [online readers]

will [generate high levels of reader engagement]

We will know this to be true when

we see [6 posts per day from each editor ]

by [3 months from now]

MVP: expert tip

Not a static thing! Commitment to a process. Experiment(s) that helps you validate (or invalidate) hypotheses about the value or growth potential for a new product over time.

(Credit to Eric Ries)

MVP: expert tip● Isolate variables ● Make the call to action 3x harder than you think it should be ● Make the time to deliver 7x faster than you think it should

be ● WRITE DOWN THE EXPECTED OUTCOME ● Say it out loud (tweet it, blog about it)● If not… email it to yourself or a co-founder, put a calendar

invite for one of your mentors

It is a BAD experiment if...#badscience

Audience Participation

what kinds of uncertainty?

Technical / product Can we build this?

Customer / market If we build this, will people use/buy it?

Business modelOnce we build this, can we find a way to make money from it?

●Technical / product ○ TESLA BATTERY, SOLAR, GOGO?○ SOME STUFF THAT AUBREY WILL TALK ABOUT IN

301

●Customer / market ○ YOUR IDEA (99% of the time)

●Business model○ GROUPON, HOMEJOY, LYFT

what kinds of uncertainty?

●Technical / product ○ usability tests, prototypes, wizard of oz

●Customer / market ○ problem interviews, landing pages,

●Business model○ landing pages, crowdfunding,

testing different kinds of uncertainty?

Pulldon't push - it's harder than it seems.

46

PASTnot future

47

48

NO SURVEYSNO SOLUTIONS

NO FOCUS GROUPSNO EXCUSESNO WOULD

NO FRIENDS NO FUTURE

NO “CONVINCING” (NEPAL:)NO B2B2BS

49

other brilliant experiments…?

assumptions: @ AirBnbNights Booked

20 Photographers

52

53

Minimum Viable Product

Translate your critical assumptions into an experiment:

1. Isolate critical assumptions for testing

2. Prioritize assumptions for testing

3. Draft your hypothesis to be tested

4. Execute an experiment (that includes MSC ahead of time)

5. Measure the results

6. Collect the data and learning in a systematic way and incorporate the learnings into new experiments

7. IF you do not do #6, 1-5 are irrelevant

validated learning using scientific principles

THINK.TEST.PROVE.Prove Wrong? Think

differentProve Right? Think BIGGER

-ALBERT EINSTEIN

Let’s get real… let’s DISCUSS the following slides for the remaining (hopefully) 30m

For startups

Why we (I) do not believe in evolution!

If you

cannot explain it simply, you

do not

UNDERSTAND IT well enough.

<<< That guy

59

If you think your customer segment is

MILLENNIALS, YOUNG PROFESSIONALS, EVERYONE, SMBs, MULTIPLE ACTORS, YOUR MENTORS, PEOPLE WHO HAVE A VESTED INTEREST IN BUILDING YOUR SOLUTION (added after the webinar) FRIENDS, FAMILY,

you do not

UNDERSTAND IT well enough.

<<< That guy

60

If you think the problem you are solving is

LACK OF TIME, LACK OF MONEY, LACK OF CUSTOMERS, LACK OF YOUR SOLUTION,

you do not

UNDERSTAND IT well enough.

<<< That guy61

“Do things that don't scale”

-Paul Graham62

You do not know HOW you did it until you KNOW how to

do it again.

63

64

1000 visitors 0 conversions

what number should you try to increase?

65

P E R F E C T I O N is achieved

NOT when there is nothing left

to A D D ,

BT WHN THRE ISNTHING LFT TO

TKE AWY

66

67

What's another word for "too early"?

68

“Science is an art not a science” -@berters

69

FACTSvs

PROBLEMSyou cannot solve a fact

learn the difference

70

UNTIL THEN

71

72

PRACTICE

73

measure results

What knowledge are you looking to gain?LEARN or PROVE

What you are going to do with the knowledge when you get it?START THE LOOP AGAIN BASED ON NEW EVIDENCE, NEW LEARNINGS, NEW PROOF!!!

why does it work?

why does it work? (less luck more skill)

path to success

“The ability to LEARN faster than your competition may be the only sustainable competitive advantage”.

Arie de Geus

“I was out of touch. But it wasn't because I didn't know enough. I just knew too much.”Gnarles Barkley

“Running down a dreamthat never would come to meworkin on a mystery, goin wherever it leads”

Tom Petty, Rock n Roller

The end (for real this time).

But really just the beginning for you. Get out there and do

not be afraid to challenge the status quo with

HYPOTHESES and DATA.

Don’t LOOK DOWN (notes, old slides, appendix, crazy experimental shit below)

1. Superforecasters - Podcast2. The Experimenter - on Netflix3. The ESPN experiment 4. LeanUX

1. Can experiments themselves be a waste 2. What is a bad experiment3. How do you set a baseline metric 4. What about a pure vision 5. What if the riskiest assumption is investment

(designing MVPs for investors)

CASH FOR EXPERIMENTS AT SXSW16

Please use the COMMENTS to comment directly in the deck on ● one slide, lesson, comment you loved, ● one that we could have improved or eliminated● We will publish the new and improved deck at the end of the week with your feedback and our learnings incorporated.

It was a privilege to speak with you all today, our doors are always open!

@adamberk + @berters