2014 experiments, tokyo
Post on 17-Oct-2014
857 views
DESCRIPTION
Deck from our February 2014 workshop in TokyoTRANSCRIPT
![Page 1: 2014 Experiments, Tokyo](https://reader033.vdocument.in/reader033/viewer/2022060106/544155e0afaf9f52208b46cd/html5/thumbnails/1.jpg)
Learn-Measure-Build Craft Useful Experiments By Running Backwards
Janice FraserFebruary 2014
![Page 2: 2014 Experiments, Tokyo](https://reader033.vdocument.in/reader033/viewer/2022060106/544155e0afaf9f52208b46cd/html5/thumbnails/2.jpg)
LUXR.CO JUNE 2012
![Page 3: 2014 Experiments, Tokyo](https://reader033.vdocument.in/reader033/viewer/2022060106/544155e0afaf9f52208b46cd/html5/thumbnails/3.jpg)
LUXR.CO JUNE 2012
TWEET!
#LeanStartup
@clevergirl@luxrco
![Page 4: 2014 Experiments, Tokyo](https://reader033.vdocument.in/reader033/viewer/2022060106/544155e0afaf9f52208b46cd/html5/thumbnails/4.jpg)
LUXR.CO JUNE 2012
• Learn the difference between an experiment and a solution test.• Discover the seven steps to crafting an experiment with a clear
result. • Practice identifying assumptions and defining testable
hypotheses. • Understand the anatomy of an experiment. • Design an experiment to test a real-life hypothesis.
• "Get out of the classroom" to run a real-world experiment on your fellow conference-goers.
• Define a result and make a go-forward decision. • Refine the experiment based on cost, time, and manpower. • Identify the four characteristics of successful experiments.
![Page 5: 2014 Experiments, Tokyo](https://reader033.vdocument.in/reader033/viewer/2022060106/544155e0afaf9f52208b46cd/html5/thumbnails/5.jpg)
© 2013 LUXr Incorporated, www.luxr.co
June & Pete’s Wedding Cake
![Page 6: 2014 Experiments, Tokyo](https://reader033.vdocument.in/reader033/viewer/2022060106/544155e0afaf9f52208b46cd/html5/thumbnails/6.jpg)
© 2013 LUXr Incorporated, www.luxr.co
http://saptstrength.com/
Pete June
![Page 7: 2014 Experiments, Tokyo](https://reader033.vdocument.in/reader033/viewer/2022060106/544155e0afaf9f52208b46cd/html5/thumbnails/7.jpg)
© 2013 LUXr Incorporated, www.luxr.co
April
http://saptstrength.com/
May
June
July
August
![Page 8: 2014 Experiments, Tokyo](https://reader033.vdocument.in/reader033/viewer/2022060106/544155e0afaf9f52208b46cd/html5/thumbnails/8.jpg)
© 2013 LUXr Incorporated, www.luxr.co
sedonaweddingcakes.com
![Page 9: 2014 Experiments, Tokyo](https://reader033.vdocument.in/reader033/viewer/2022060106/544155e0afaf9f52208b46cd/html5/thumbnails/9.jpg)
© 2013 LUXr Incorporated, www.luxr.co
freebestpictures.blogspot.com
![Page 10: 2014 Experiments, Tokyo](https://reader033.vdocument.in/reader033/viewer/2022060106/544155e0afaf9f52208b46cd/html5/thumbnails/10.jpg)
© 2013 LUXr Incorporated, www.luxr.co
thekitchn.com
![Page 11: 2014 Experiments, Tokyo](https://reader033.vdocument.in/reader033/viewer/2022060106/544155e0afaf9f52208b46cd/html5/thumbnails/11.jpg)
© 2013 LUXr Incorporated, www.luxr.co
Zoe Clark, The Cake Parlour
![Page 12: 2014 Experiments, Tokyo](https://reader033.vdocument.in/reader033/viewer/2022060106/544155e0afaf9f52208b46cd/html5/thumbnails/12.jpg)
© 2013 LUXr Incorporated, www.luxr.co
sedonaweddingcakes.com
Zoe Clark, The Cake Parlour
![Page 13: 2014 Experiments, Tokyo](https://reader033.vdocument.in/reader033/viewer/2022060106/544155e0afaf9f52208b46cd/html5/thumbnails/13.jpg)
© 2013 LUXr Incorporated, www.luxr.co
LEAN Star tup
![Page 14: 2014 Experiments, Tokyo](https://reader033.vdocument.in/reader033/viewer/2022060106/544155e0afaf9f52208b46cd/html5/thumbnails/14.jpg)
© LUXR.CO APRIL 2013
LEAN Star tup
![Page 15: 2014 Experiments, Tokyo](https://reader033.vdocument.in/reader033/viewer/2022060106/544155e0afaf9f52208b46cd/html5/thumbnails/15.jpg)
© LUXR.CO APRIL 2013
Build measure
learn!
![Page 16: 2014 Experiments, Tokyo](https://reader033.vdocument.in/reader033/viewer/2022060106/544155e0afaf9f52208b46cd/html5/thumbnails/16.jpg)
© LUXR.CO APRIL 2013
![Page 17: 2014 Experiments, Tokyo](https://reader033.vdocument.in/reader033/viewer/2022060106/544155e0afaf9f52208b46cd/html5/thumbnails/17.jpg)
© LUXR.CO 2013
Lean Startup is NOT
Cheap Startup
Fast Startup
Shortcut Startup
![Page 18: 2014 Experiments, Tokyo](https://reader033.vdocument.in/reader033/viewer/2022060106/544155e0afaf9f52208b46cd/html5/thumbnails/18.jpg)
© LUXR.CO APRIL 2013
An approach for building companies that are creating new products and services in situations of extreme uncertainty.
The approach advocates creating small products that test the entrepreneurʼs assumptions, and using customer feedback to evolve the product, thereby reducing waste.
Lean Startup is...
![Page 19: 2014 Experiments, Tokyo](https://reader033.vdocument.in/reader033/viewer/2022060106/544155e0afaf9f52208b46cd/html5/thumbnails/19.jpg)
© LUXR.CO APRIL 2013
1. List your assumptions.2. Understand your customers.3. Experiment efficiently.4. Adjust direction based on evidence.
![Page 20: 2014 Experiments, Tokyo](https://reader033.vdocument.in/reader033/viewer/2022060106/544155e0afaf9f52208b46cd/html5/thumbnails/20.jpg)
© LUXR.CO APRIL 2013
1. List your assumptions.2. Understand your customers.3. Experiment efficiently.4. Adjust direction based on evidence.
![Page 21: 2014 Experiments, Tokyo](https://reader033.vdocument.in/reader033/viewer/2022060106/544155e0afaf9f52208b46cd/html5/thumbnails/21.jpg)
© LUXR.CO APRIL 2013
Go backward to go forward.
Learn
Measure
Build
![Page 22: 2014 Experiments, Tokyo](https://reader033.vdocument.in/reader033/viewer/2022060106/544155e0afaf9f52208b46cd/html5/thumbnails/22.jpg)
© LUXR.CO 2013
Plot the difference
THINK
release
MAKErelease
MAKErelease
MAKE
RISK
= U
NVAL
IDAT
ED E
FFOR
T
TIME
![Page 23: 2014 Experiments, Tokyo](https://reader033.vdocument.in/reader033/viewer/2022060106/544155e0afaf9f52208b46cd/html5/thumbnails/23.jpg)
© LUXR.CO 2013
Lots of little wigglesRI
SK =
UNV
ALID
ATED
EFF
ORT
TIME
![Page 24: 2014 Experiments, Tokyo](https://reader033.vdocument.in/reader033/viewer/2022060106/544155e0afaf9f52208b46cd/html5/thumbnails/24.jpg)
© LUXR.CO 2013
RISK
= U
NVAL
IDAT
ED E
FFOR
T
TIME
Each wiggle is a learning cycle.
MAKE
releaseBUILD
LEARN
BUILD
MEASURE
![Page 25: 2014 Experiments, Tokyo](https://reader033.vdocument.in/reader033/viewer/2022060106/544155e0afaf9f52208b46cd/html5/thumbnails/25.jpg)
LUXR.CO JUNE 2012
There’s a pattern to startup work:
ideation decisionmaking
ideation etc.
![Page 26: 2014 Experiments, Tokyo](https://reader033.vdocument.in/reader033/viewer/2022060106/544155e0afaf9f52208b46cd/html5/thumbnails/26.jpg)
LUXR.CO JUNE 2012
In foolproof experiments, the
results should be self-evident.
![Page 27: 2014 Experiments, Tokyo](https://reader033.vdocument.in/reader033/viewer/2022060106/544155e0afaf9f52208b46cd/html5/thumbnails/27.jpg)
LUXR.CO JUNE 2012
1. Choose the problem to tackle. Clarify what’s uncertain, what you need to learn.
2. Understand the possible answers.3. Brainstorm “what you could do to prove it”.4. Choose the most broadly differentiated set of
indicators.5. Run the experiment. 6. Write down the results...on a sheet of paper. 7. Project the results into your future.
7 steps to foolproof experiments
![Page 28: 2014 Experiments, Tokyo](https://reader033.vdocument.in/reader033/viewer/2022060106/544155e0afaf9f52208b46cd/html5/thumbnails/28.jpg)
LUXR.CO JUNE 2012
The Workshop
![Page 29: 2014 Experiments, Tokyo](https://reader033.vdocument.in/reader033/viewer/2022060106/544155e0afaf9f52208b46cd/html5/thumbnails/29.jpg)
LUXR.CO JUNE 2012
Hypothesis
We believe that modern families would benefit from having better ways to store their
belongings, and are willing to pay for a service that will pick up and store boxes, then deliver
them back home on demand.
Today’s Challenge
![Page 30: 2014 Experiments, Tokyo](https://reader033.vdocument.in/reader033/viewer/2022060106/544155e0afaf9f52208b46cd/html5/thumbnails/30.jpg)
LUXR.CO JUNE 2012
Box Me UpProvides you with plastic storage boxesTransports your stuff to storagePay to get your stuff back Per-box per year2800¥ per box per yearoutside of the city (less expensive place)box size: 30 gallon boxservice area is in Tokyoinsurance is 20,000 yen per boxtemperature and humidity are normally conditionedno limit for storage perioe24 hours access to the boxprivately storagepayment method: paypal, square, credit card, no cashdelivery will be by carnothing illegal, perishable, waste will be stored3 employees in the companymain customer will be familiesgot 70,000,000 yen for initial investment
![Page 31: 2014 Experiments, Tokyo](https://reader033.vdocument.in/reader033/viewer/2022060106/544155e0afaf9f52208b46cd/html5/thumbnails/31.jpg)
LUXR.CO JUNE 2012
Experiment Framework
For Example:
We believe people like [customer type] have a need for(or problem doing) [need/action/behavior].
The smallest thing we can do to prove that need is [experiment].
We will know we have succeeded when [quantitative/measurable outcome] or [qualitative/observable outcome].
Every experiment has three parts:
1. Hypothesis that is provable/disprovable
2. The experiment itself; the thing you build
3. An indicator of result
![Page 32: 2014 Experiments, Tokyo](https://reader033.vdocument.in/reader033/viewer/2022060106/544155e0afaf9f52208b46cd/html5/thumbnails/32.jpg)
LUXR.CO JUNE 2012
{Activity}
Identify assumptions10 assumptions, 1 per sticky
![Page 33: 2014 Experiments, Tokyo](https://reader033.vdocument.in/reader033/viewer/2022060106/544155e0afaf9f52208b46cd/html5/thumbnails/33.jpg)
LUXR.CO JUNE 2012
{Activity}
Divide into 2 pilesWill kill the company in the next 6 months
if weʼre wrong
Everything else
![Page 34: 2014 Experiments, Tokyo](https://reader033.vdocument.in/reader033/viewer/2022060106/544155e0afaf9f52208b46cd/html5/thumbnails/34.jpg)
LUXR.CO JUNE 2012
{Activity}
Review & de-dupe
![Page 35: 2014 Experiments, Tokyo](https://reader033.vdocument.in/reader033/viewer/2022060106/544155e0afaf9f52208b46cd/html5/thumbnails/35.jpg)
LUXR.CO JUNE 2012
{Activity}
Stack rank the urgents
![Page 36: 2014 Experiments, Tokyo](https://reader033.vdocument.in/reader033/viewer/2022060106/544155e0afaf9f52208b46cd/html5/thumbnails/36.jpg)
LUXR.CO JUNE 2012
{Activity}
Pick the top one
![Page 37: 2014 Experiments, Tokyo](https://reader033.vdocument.in/reader033/viewer/2022060106/544155e0afaf9f52208b46cd/html5/thumbnails/37.jpg)
LUXR.CO JUNE 2012
{Activity}
Design an experiment to learn if this is true.
we believe that ________________
_________________________________
_________________________________
_________________________________
_________________________________
_________________________________
_________________________________
__________________________
Briefly describe it.
State how you will know if the hypothesis is valid or invalid. This can be quantitative evidence or qualitative.
How much time/money/effort will it take?
![Page 38: 2014 Experiments, Tokyo](https://reader033.vdocument.in/reader033/viewer/2022060106/544155e0afaf9f52208b46cd/html5/thumbnails/38.jpg)
LUXR.CO JUNE 2012
{Activity}
Discuss, then pick the experiment to run.
we believe that ________________
__________________________________
__________________________________
__________________________________
__________________________________
__________________________________
__________________________________
____________ in 2 days.
Decide
we believe that ________________
__________________________________
__________________________________
__________________________________
__________________________________
__________________________________
__________________________________
____________ in 2 weeks.
we believe that ________________
__________________________________
__________________________________
__________________________________
__________________________________
__________________________________
__________________________________
____________ in 2 months.
✔
![Page 39: 2014 Experiments, Tokyo](https://reader033.vdocument.in/reader033/viewer/2022060106/544155e0afaf9f52208b46cd/html5/thumbnails/39.jpg)
LUXR.CO JUNE 2012
Experiments that were designed
assumption experiment measurable/observable outcome
busy people need a service to help them getmenial stuff done
fb ad, neighborhood targeting presents an offer, landing page. Phone number or tweet. No out
of pocket cost.
10% ctr on ad, 10% ctr on landing page. 2 days
busy people need a service to help them getmenial stuff done
find busy people (in offices). Set up a “shop”. Sit in their reception and see if we can get jobs. Prevail upon friend
who runs 300-person company
5% of employees will make a hire. 1 repeat booking
2 wks
busy people need a service to help them getmenial stuff done
ad in 10 different offices, diff types of companies 2 weeks.
![Page 40: 2014 Experiments, Tokyo](https://reader033.vdocument.in/reader033/viewer/2022060106/544155e0afaf9f52208b46cd/html5/thumbnails/40.jpg)
LUXR.CO JUNE 2012
What did you learn?
Top 5 things
{Debrief}
![Page 41: 2014 Experiments, Tokyo](https://reader033.vdocument.in/reader033/viewer/2022060106/544155e0afaf9f52208b46cd/html5/thumbnails/41.jpg)
LUXR.CO JUNE 2012
Design the next experiment:
2 days, 2 weeks, 2 months
{Next Steps}
![Page 42: 2014 Experiments, Tokyo](https://reader033.vdocument.in/reader033/viewer/2022060106/544155e0afaf9f52208b46cd/html5/thumbnails/42.jpg)
LUXR.CO JUNE 2012
What’s the right size for the experiment?
{Choose}
![Page 43: 2014 Experiments, Tokyo](https://reader033.vdocument.in/reader033/viewer/2022060106/544155e0afaf9f52208b46cd/html5/thumbnails/43.jpg)
LUXR.CO JUNE 2012
Techniques Checklist• Brainstorm quietly• Use Sticky Notes for ideas• Self-Edit• Go to the wall for team sharing• Stack-ranking• Roman Voting• Right-Sizing the work
![Page 44: 2014 Experiments, Tokyo](https://reader033.vdocument.in/reader033/viewer/2022060106/544155e0afaf9f52208b46cd/html5/thumbnails/44.jpg)
LUXR.CO JUNE 2012
FISHBOWL