FAKE DOOR TESTING How to Test Product Ideas Quickly

Jess Lee, CEO & Co-founder, Polyvore

Jess Lee HONG KONG /

STANFORD CS 2004 /

GOOGLE MAPS PM /

POLYVORE CEO /

DRAWING / BACON /

SCI FI & ACTION MOVIES /

jessyoko.com @jesskah jess.polyvore.com

The Evolution of Shopping INFORMATION

SPECS / FACTUAL REVIEWS / PRICE INSPIRATION

TASTE / TRENDS / ASPIRATION

Mix & match products you love • Shop looks created by tastemakers

POLYVORE.COM

POLYVORE APP

The Great Startup Rollercoaster

TIME

EFFO

RT

/ A

NX

IETY

1ST HIRE

FUNDRAISING FIRST N USERS

PRODUCT-MARKET FIT

LIKELIHOOD OF HEAVY DRINKING

SOURCE: HTTP://BLOG.ELADGIL.COM/2010/09/ARE-YOU-ALCOHOLIC-YET-OR-GREAT-STARTUP.HTML

Product-Market Fit Being in a good market with a product that can satisfy that market

Market Whose problem are you trying to solve? How many people have this problem?

Product What solution will best serve this market?

How can I build this solution?

Ask friends / family

CONVENTIONAL WISDOM

Talk to Users

Ask Industry Experts / Entrepreneurs / VCs

CONVENTIONAL WISDOM

Talk to Users

THE UNFORTUNATE TRUTH

People are bad at predicting what they want

People are good at reacting to things that exist

THE SAVING GRACE

THE ONLY SOLUTION

Build & test it quickly

HOW TO BUILD & TEST QUICKLY

Fake Door Testing

Only build the important bits. Fake the rest. Test on real users.

HOW TO BUILD & TEST QUICKLY

Fake Door Testing

EXAMPLE #1

App Notifications

EXAMPLE #2

Universal Shopping Cart

EXAMPLE #3

App Internationalization

EXAMPLE #4

Outfit Sales

List key assumptions

Identify the biggest unknowns

Build the unknowns, fake the rest

Measure your success

Steps for Fake Doors

•  People will buy more items if they get a bigger discount

•  We can establish partnerships with retailers for this idea

List Key Assumptions IDEA: Shop user-generated outfits and get a sliding discount based on the # of items you buy

•  People are interested in shopping outfits

•  We can scalably create outfits with products from partners

ASSUMPTIONS:

•  We can handle payment, fulfillment, shipping, returns

Identify Biggest Unknowns

•  We can establish partnerships with retailers for this idea

•  We can scalably create outfits with products from partners

•  People will buy more items if they get a bigger discount

•  People are interested in shopping outfits

BIGGEST UNKNOWNS:

•  We can handle payment, fulfillment, shipping, returns

SEEMS LIKE A SOLVABLE PROBLEM:

•  We can establish partnerships with retailers for this idea

•  We can scalably create outfits with products from partners

•  People will buy more items if they get a bigger discount

•  People are interested in shopping outfits

BIGGEST UNKNOWNS:

•  We can handle payment, fulfillment, shipping, returns

SEEMS LIKE A SOLVABLE PROBLEM:

Build the Unknowns, Fake the Rest

Build outfit sale UI

Handle shipping ourselves

Show it to real users

Fake the partners

Make the outfits ourselves

Build outfit sales UI

Show it to users

Fake the Partners

Make the outfits ourselves

Handle shipping, fulfillment, returns ourselves

Measure Your Success RESULTS AFTER 3 MONTHS, 63 OUTFITS, $3722.16: •  1.89% conversion rate

•  2.93% checkout rate

•  2.32 items / order

•  21% return rate

LESSONS LEARNED

•  People will buy more if the discount goes up

•  Our checkout flow with Amazon Payments is terrible

•  Needs further improvements to be highly profitable

Need to be tolerant of failure

Define clear metrics & constraints

Don’t betray your users’ trust

Not a replacement for ideation

Tips for Fake Doors

Tolerance for Failure

Clear Metrics & Constraints

Don’t Betray Users’ Trust

Not a Replacement for Ideation

Thank You!