lean analytics for startups - leweb2010

99
Lean analytics for startups Wednesday, December 8, 2010

Post on 18-Sep-2014

13 views

Category:

Technology


2 download

DESCRIPTION

20-minute speed-run presentation on what metrics and web analytics information startups need to collect. Focuses on companies with a lean methodology, and the kinds of data that will actually help them achieve product/market fit before the money runs out.

TRANSCRIPT

Page 1: Lean analytics for startups - Leweb2010

Lean analytics for startups

Wednesday, December 8, 2010

Page 2: Lean analytics for startups - Leweb2010

@acroll(if you have questions)

Wednesday, December 8, 2010

Page 3: Lean analytics for startups - Leweb2010

Wednesday, December 8, 2010

Page 4: Lean analytics for startups - Leweb2010

Wednesday, December 8, 2010

Page 5: Lean analytics for startups - Leweb2010

Lean Not lean

Wednesday, December 8, 2010

Page 6: Lean analytics for startups - Leweb2010

Ries, Mclure, and Blankare often misquoted.

Wednesday, December 8, 2010

Page 7: Lean analytics for startups - Leweb2010

They never said “fail faster.”

Wednesday, December 8, 2010

Page 8: Lean analytics for startups - Leweb2010

Instead:“Learn and adapt.”

Wednesday, December 8, 2010

Page 9: Lean analytics for startups - Leweb2010

Waterfall, agile, and leanThree approaches for three situations.

Wednesday, December 8, 2010

Page 10: Lean analytics for startups - Leweb2010

Waterfall approachYou know the problem and the solution.

Wednesday, December 8, 2010

Page 11: Lean analytics for startups - Leweb2010

Known set of requirements

Known ways to satisfy them

Spec Build Test Launch

Wednesday, December 8, 2010

Page 12: Lean analytics for startups - Leweb2010

Agile methodologiesKnow the problem, find the solution

Wednesday, December 8, 2010

Page 13: Lean analytics for startups - Leweb2010

Known set of requirements

Unclear how to satisfy them

Build Test LaunchViable?Problem

statement

Adjust

Sprints

Unknown set of requirements

Wednesday, December 8, 2010

Page 14: Lean analytics for startups - Leweb2010

Most startupsDon’t even know what problem they solve.

Wednesday, December 8, 2010

Page 15: Lean analytics for startups - Leweb2010

Possible problem space

Possible viable

offeringTrial startup

Possible viable

offeringTrial startup

Possible viable offering

Trial startup

Possible viable offering

Trial startup

You are here

PIVOT

Wednesday, December 8, 2010

Page 16: Lean analytics for startups - Leweb2010

All startups are lean at first.Which has some interesting consequences.

Wednesday, December 8, 2010

Page 17: Lean analytics for startups - Leweb2010

Getting features right matters less.

Knowing what’s not working, quickly, matters more.

Wednesday, December 8, 2010

Page 18: Lean analytics for startups - Leweb2010

“We should” matters less.

“What if” matters more.

Wednesday, December 8, 2010

Page 19: Lean analytics for startups - Leweb2010

Your roadmap matters less.

Parallel experiments matter more.

Wednesday, December 8, 2010

Page 20: Lean analytics for startups - Leweb2010

User groups and surveys matter less.

Real-world adoption & usage analysis matters more.

Wednesday, December 8, 2010

Page 21: Lean analytics for startups - Leweb2010

Business plans matter less.

Business models matter more.

Wednesday, December 8, 2010

Page 22: Lean analytics for startups - Leweb2010

http://www.flickr.com/photos/tinfoilraccoon/197640807/Wednesday, December 8, 2010

Page 23: Lean analytics for startups - Leweb2010

Cost

Revenue Lemonade( ) x Units sold + tips

Lemon + cups + sugar( ) x Units made

Markers + cardboard( ) x Signs made

Staffing( ) x Hours worked

Profit!Wednesday, December 8, 2010

Page 24: Lean analytics for startups - Leweb2010

Wednesday, December 8, 2010

Page 25: Lean analytics for startups - Leweb2010

http://www.flickr.com/photos/ethanhein/3207831470/#/

(maybe not that fun a game)

Wednesday, December 8, 2010

Page 26: Lean analytics for startups - Leweb2010

Clouds and analyticsIt’s never been a better time to start a company.

Wednesday, December 8, 2010

Page 27: Lean analytics for startups - Leweb2010

Clouds make experimenting easy, avoid upfront dilution.

Analytics make experimenting smart & fast.

Wednesday, December 8, 2010

Page 28: Lean analytics for startups - Leweb2010

http://www.flickr.com/photos/ecastro/3053916892/

I love to talk about clouds(but not today)

Wednesday, December 8, 2010

Page 29: Lean analytics for startups - Leweb2010

I’m going to focus on the other partWhat to watch on your website, if you’re a startup.

Wednesday, December 8, 2010

Page 30: Lean analytics for startups - Leweb2010

So what could we watch?

Wednesday, December 8, 2010

Page 31: Lean analytics for startups - Leweb2010

Analytics(what did they

do on the site?)

Competition(what are they

up to?)

Performability(could they do

what they wanted to?)

VoC(what were

their motivations?)

Usability(how did they interact with

it?)

Community(what were

they saying?)

“Hard” data

“Soft” data

Complete Web Monitoring

Wednesday, December 8, 2010

Page 32: Lean analytics for startups - Leweb2010

http://www.flickr.com/photos/itsgreg/446061432/

Analytics is the measurement of movement towards your business goals.

Wednesday, December 8, 2010

Page 33: Lean analytics for startups - Leweb2010

NEWVISITORSGROWTH

BOUNCERATE

LOSS

CONVERSIONRATE

GOALVALUE

xTIME

ONSITE

PAGESPER

VISIT

NUMBEROF VISITS

SEARCHES

TWEETS

MENTIONS

ADS SEEN

ATTENTION ENGAGEMENT CONVERSION

Wednesday, December 8, 2010

Page 34: Lean analytics for startups - Leweb2010

Conversions

Visits

Shopping cart

Payment options

Wednesday, December 8, 2010

Page 35: Lean analytics for startups - Leweb2010

Conversions

Visits

Shopping cart

Payment options

KPIsC

onversionrateBounce

rate

Wednesday, December 8, 2010

Page 36: Lean analytics for startups - Leweb2010

http

://w

ww

.flic

kr.c

om/p

hoto

s/m

rmoo

rey/

1606

5423

6

Wednesday, December 8, 2010

Page 37: Lean analytics for startups - Leweb2010

http://www.flickr.com/photos/duncan/1252272164/Wednesday, December 8, 2010

Page 38: Lean analytics for startups - Leweb2010

http://www.flickr.com/photos/intherough/3573333256/Wednesday, December 8, 2010

Page 39: Lean analytics for startups - Leweb2010

http://www.flickr.com/photos/jetheriot/648950773/Wednesday, December 8, 2010

Page 40: Lean analytics for startups - Leweb2010

Wednesday, December 8, 2010

Page 41: Lean analytics for startups - Leweb2010

Wednesday, December 8, 2010

Page 42: Lean analytics for startups - Leweb2010

Wednesday, December 8, 2010

Page 43: Lean analytics for startups - Leweb2010

Wednesday, December 8, 2010

Page 44: Lean analytics for startups - Leweb2010

Wednesday, December 8, 2010

Page 45: Lean analytics for startups - Leweb2010

Wednesday, December 8, 2010

Page 46: Lean analytics for startups - Leweb2010

Wednesday, December 8, 2010

Page 47: Lean analytics for startups - Leweb2010

Wednesday, December 8, 2010

Page 48: Lean analytics for startups - Leweb2010

Wednesday, December 8, 2010

Page 49: Lean analytics for startups - Leweb2010

Wednesday, December 8, 2010

Page 50: Lean analytics for startups - Leweb2010

Impact of page load time on average daily searches per user

-0.60%

-0.45%

-0.30%

-0.15%

0%

50ms pre-header 100ms pre-header 200ms post-header 200ms post-ad 400ms post-header

Wednesday, December 8, 2010

Page 51: Lean analytics for startups - Leweb2010

Wednesday, December 8, 2010

Page 52: Lean analytics for startups - Leweb2010

Impact of additional delay on business metrics

-5.00%

-3.75%

-2.50%

-1.25%

0%

50 200 500 1000 2000Queries/visitor Query refinement Revenue/visitorAny clicks Satisfaction

Wednesday, December 8, 2010

Page 53: Lean analytics for startups - Leweb2010

Shopzilla’s makeover

Wednesday, December 8, 2010

Page 54: Lean analytics for startups - Leweb2010

5-12%Wednesday, December 8, 2010

Page 55: Lean analytics for startups - Leweb2010

Wednesday, December 8, 2010

Page 56: Lean analytics for startups - Leweb2010

Traffic levels

0

2,250

4,500

6,750

9,000

Optimized Unoptimized

4,740

8,505

Tota

l num

ber o

f visi

ts

Visitor experience

Wednesday, December 8, 2010

Page 57: Lean analytics for startups - Leweb2010

Bounce rate

0

5

10

15

20

Optimized Unoptimized

14.35%13.38%

Visit

s tha

t bou

nced

Visitor experience

Wednesday, December 8, 2010

Page 58: Lean analytics for startups - Leweb2010

Average time on site

0

8

16

23

31

Optimized Unoptimized

23.8330.17

Time o

n site

(min

utes

)

Visitor experience

Wednesday, December 8, 2010

Page 59: Lean analytics for startups - Leweb2010

Pages per visit

0

4

8

12

16

Optimized Unoptimized

11.04

15.64

Aver

age p

ages

seen

Visitor experience

Wednesday, December 8, 2010

Page 60: Lean analytics for startups - Leweb2010

Conversion rateand order value

0

5

10

15

20

Conversion rate Order value

5.51

16.07

Diffe

renc

e due

to op

timiza

tion

Wednesday, December 8, 2010

Page 61: Lean analytics for startups - Leweb2010

Community management

Hugging

Bunnies

Retweets

Friends!

Wednesday, December 8, 2010

Page 62: Lean analytics for startups - Leweb2010

Conversions

Visits

Shopping cart

Payment options

Unpaid search Communitymentions

Emailcampaign Banner ad

• Google PageRank• Sessions-to-clicks

ratio

• Cost of ads(CPM)

• Clickthroughrate

• Open rate• Opt-out rate?

Wednesday, December 8, 2010

Page 63: Lean analytics for startups - Leweb2010

But there’s a danger:vanity metrics.

Wednesday, December 8, 2010

Page 64: Lean analytics for startups - Leweb2010

Wednesday, December 8, 2010

Page 65: Lean analytics for startups - Leweb2010

So what should you watch?The lean analytics shortlist.

Wednesday, December 8, 2010

Page 66: Lean analytics for startups - Leweb2010

In a startup, the purpose of analytics is to iterate to a product/market fit before the money runs out.

If you remember only one thing...

Wednesday, December 8, 2010

Page 67: Lean analytics for startups - Leweb2010

Message reach

Infrastructure health

Market sentiment

User engagement

Business model

Sustainability

Viral coefficient

Amplification

Performability

Customer research

Community sentiment

Time since last visit

Core goals

Extended funnel abandonment

Movement towards goals

Cost per engaged visitor

Peak to average ratio

Minimum sustainable burn

Wednesday, December 8, 2010

Page 68: Lean analytics for startups - Leweb2010

Viral coefficientOn average, for each message they hear, how many people does someone tell?

1Wednesday, December 8, 2010

Page 69: Lean analytics for startups - Leweb2010

------------------------------------------------------Get your free private email at http://www.hotmail.com------------------------------------------------------

Wednesday, December 8, 2010

Page 70: Lean analytics for startups - Leweb2010

!Wednesday, December 8, 2010

Page 71: Lean analytics for startups - Leweb2010

v ≠ 1, pt  = δp0 (1 – vt+1) / (1 – v) + p0 

http://robert.zubek.net/blog/2008/01/30/viral-coefficient-calculation/Wednesday, December 8, 2010

Page 72: Lean analytics for startups - Leweb2010

How well are messages being amplified?

2Wednesday, December 8, 2010

Page 73: Lean analytics for startups - Leweb2010

50% 75% 100% 50% = 4

2 3 1 = 6

14.5

Downstream reach (aggregate followers)

Amplification chance(how relevant the content

is + what the recipient thinks of the sender)

25%

.5

50%

2

50%

1 = 3.5

Wednesday, December 8, 2010

Page 74: Lean analytics for startups - Leweb2010

Repurposing(spread to othercommunities)

Amplification(virality andmessage spread)

Conversions

Visits

Shopping cart

Payment options

Seed (startingcommunity)

Reach (impressions)

Wednesday, December 8, 2010

Page 75: Lean analytics for startups - Leweb2010

Viral message spread

Conversions

Visits

Shopping cart

Payment options

Reach (impressions) Emphasis on

getting viral ratio above 1(Retweeting, Fan, Email

forward, Reddit upvote, other

loops)

Wednesday, December 8, 2010

Page 76: Lean analytics for startups - Leweb2010

Megablogger proponents

Conversions

Visits

Shopping cart

Payment options

Seed (startingcommunity)

Reach (impressions)

Emphasis on convincing highly-followed, highly

acted-upon seed group to spread

the word.

Wednesday, December 8, 2010

Page 77: Lean analytics for startups - Leweb2010

A call to action

Conversions

Visits

Shopping cart

Payment options

Reach (impressions)

Emphasis on maximizing

impression-to-click ratio within the community

Wednesday, December 8, 2010

Page 78: Lean analytics for startups - Leweb2010

Goal attainment: funnel

700,000s

1,642

150RT

327

1015

x $100x $20

x $7

20Seed ratio:

35,000:1

Followers

Visitors:0.23%

Conversions:1.95%

Revenues:Average: $39.54Median: $20

Total: $1,005

Amplification: 2.9%

2 Repurposing

2,000

Wednesday, December 8, 2010

Page 79: Lean analytics for startups - Leweb2010

How performable are we?Health of the infrastructure (uptime, latency)

3Wednesday, December 8, 2010

Page 80: Lean analytics for startups - Leweb2010

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16

0

5,000

Uniq

ue p

age

view

s

>4s or error

2-4s

<2s

Conversions

$0

$10,000

Revenue (total sales)

Wednesday, December 8, 2010

Page 81: Lean analytics for startups - Leweb2010

What do visitors think of us?Both on the site (VoC) and in the world (community monitoring)

4&5Wednesday, December 8, 2010

Page 82: Lean analytics for startups - Leweb2010

Wednesday, December 8, 2010

Page 83: Lean analytics for startups - Leweb2010

Wednesday, December 8, 2010

Page 84: Lean analytics for startups - Leweb2010

How engaged are users?Time since last visit, as a histogram, not an average, compared to baselines.

6Wednesday, December 8, 2010

Page 85: Lean analytics for startups - Leweb2010

Days since last visit

1200

1000

800

600

400

200

01 2 3 4 5 6 7 8 9

Days since last engagement

January February

Disengaged(>10 days)

25000

20000

15000

10000

5000

0

Num

ber o

f use

rs

Wednesday, December 8, 2010

Page 86: Lean analytics for startups - Leweb2010

Core goals driving the businessPick the 3-4 tasks you’ve set for users. Always know what these are and why they correlate to business growth.

7Wednesday, December 8, 2010

Page 87: Lean analytics for startups - Leweb2010

Cost

Revenue Lemonade( ) x Units sold + tips

Lemon + cups + sugar( ) x Units made

Markers + cardboard( ) x Signs made

Staffing( ) x Hours worked

Profit!

(Remember the model?)

Wednesday, December 8, 2010

Page 88: Lean analytics for startups - Leweb2010

Extended funnel abandonmentNot just on the site: Email open rate

8Wednesday, December 8, 2010

Page 89: Lean analytics for startups - Leweb2010

Wednesday, December 8, 2010

Page 90: Lean analytics for startups - Leweb2010

In the last build, what moved us towards & away from goals?Requires knowing the goals and the key features added

9Wednesday, December 8, 2010

Page 91: Lean analytics for startups - Leweb2010

Cost per visitorOperations costs per site visitor

10Wednesday, December 8, 2010

Page 92: Lean analytics for startups - Leweb2010

(this had better be greater than the revenues and conversion rate.)

Wednesday, December 8, 2010

Page 93: Lean analytics for startups - Leweb2010

Peak-to-average ratioHow much busier are you at busy times?

What premium do you pay for that surplus?

Can you decommission well?

11Wednesday, December 8, 2010

Page 94: Lean analytics for startups - Leweb2010

http://community.microsoftadvertising.comWednesday, December 8, 2010

Page 95: Lean analytics for startups - Leweb2010

Wednesday, December 8, 2010

Page 96: Lean analytics for startups - Leweb2010

Minimum sustainable burn

12Wednesday, December 8, 2010

Page 97: Lean analytics for startups - Leweb2010

http://www.flickr.com/photos/dougww/833127501/Wednesday, December 8, 2010

Page 98: Lean analytics for startups - Leweb2010

Message reach

Infrastructure health

Market sentiment

User engagement

Business model

Sustainability

Viral coefficient

Amplification

Performability

Customer research

Community sentiment

Time since last visit

Core goals

Extended funnel abandonment

Movement towards goals

Cost per engaged visitor

Peak to average ratio

Minimum sustainable burn

12thingsto

watch

Wednesday, December 8, 2010