lean analytics for startups - leweb2010

Lean analytics for startups

Wednesday, December 8, 2010

@acroll(if you have questions)

Lean Not lean

Ries, Mclure, and Blankare often misquoted.

They never said “fail faster.”

Instead:“Learn and adapt.”

Waterfall, agile, and leanThree approaches for three situations.

Waterfall approachYou know the problem and the solution.

Known set of requirements

Known ways to satisfy them

Spec Build Test Launch

Agile methodologiesKnow the problem, find the solution

Known set of requirements

Unclear how to satisfy them

Build Test LaunchViable?Problem

statement

Adjust

Sprints

Unknown set of requirements

Most startupsDon’t even know what problem they solve.

Possible problem space

Possible viable

offeringTrial startup

Possible viable

offeringTrial startup

Possible viable offering

Trial startup

Possible viable offering

Trial startup

You are here

All startups are lean at first.Which has some interesting consequences.

Getting features right matters less.

Knowing what’s not working, quickly, matters more.

“We should” matters less.

“What if” matters more.

Your roadmap matters less.

Parallel experiments matter more.

User groups and surveys matter less.

Real-world adoption & usage analysis matters more.

Business plans matter less.

Business models matter more.

http://www.flickr.com/photos/tinfoilraccoon/197640807/Wednesday, December 8, 2010

Revenue Lemonade( ) x Units sold + tips

Lemon + cups + sugar( ) x Units made

Markers + cardboard( ) x Signs made

Staffing( ) x Hours worked

Profit!Wednesday, December 8, 2010

http://www.flickr.com/photos/ethanhein/3207831470/#/

(maybe not that fun a game)

Clouds and analyticsIt’s never been a better time to start a company.

Clouds make experimenting easy, avoid upfront dilution.

Analytics make experimenting smart & fast.

http://www.flickr.com/photos/ecastro/3053916892/

I love to talk about clouds(but not today)

I’m going to focus on the other partWhat to watch on your website, if you’re a startup.

So what could we watch?

Analytics(what did they

do on the site?)

Competition(what are they

up to?)

Performability(could they do

what they wanted to?)

VoC(what were

their motivations?)

Usability(how did they interact with

Community(what were

they saying?)

“Hard” data

“Soft” data

Complete Web Monitoring

http://www.flickr.com/photos/itsgreg/446061432/

Analytics is the measurement of movement towards your business goals.

NEWVISITORSGROWTH

BOUNCERATE

CONVERSIONRATE

GOALVALUE

ONSITE

PAGESPER

NUMBEROF VISITS

SEARCHES

TWEETS

MENTIONS

ADS SEEN

ATTENTION ENGAGEMENT CONVERSION

Conversions

Visits

Shopping cart

Payment options

Conversions

Visits

Shopping cart

Payment options

onversionrateBounce

http://www.flickr.com/photos/duncan/1252272164/Wednesday, December 8, 2010

http://www.flickr.com/photos/intherough/3573333256/Wednesday, December 8, 2010

http://www.flickr.com/photos/jetheriot/648950773/Wednesday, December 8, 2010

Impact of page load time on average daily searches per user

-0.60%

-0.45%

-0.30%

-0.15%

50ms pre-header 100ms pre-header 200ms post-header 200ms post-ad 400ms post-header

Impact of additional delay on business metrics

-5.00%

-3.75%

-2.50%

-1.25%

50 200 500 1000 2000Queries/visitor Query refinement Revenue/visitorAny clicks Satisfaction

Shopzilla’s makeover

5-12%Wednesday, December 8, 2010

Traffic levels

Optimized Unoptimized

f visi

Visitor experience

Bounce rate

14.35%13.38%

Visitor experience

Average time on site

23.8330.17

Time o

n site

Visitor experience

Pages per visit

Visitor experience

Conversion rateand order value

Conversion rate Order value

timiza

Community management

Hugging

Bunnies

Retweets

Friends!

Conversions

Visits

Shopping cart

Payment options

Unpaid search Communitymentions

Emailcampaign Banner ad

• Google PageRank• Sessions-to-clicks

• Cost of ads(CPM)

• Clickthroughrate

• Open rate• Opt-out rate?

But there’s a danger:vanity metrics.

So what should you watch?The lean analytics shortlist.

In a startup, the purpose of analytics is to iterate to a product/market fit before the money runs out.

If you remember only one thing...

Message reach

Infrastructure health

Market sentiment

User engagement

Business model

Sustainability

Viral coefficient

Amplification

Performability

Customer research

Community sentiment

Time since last visit

Core goals

Extended funnel abandonment

Movement towards goals

Cost per engaged visitor

Peak to average ratio

Minimum sustainable burn

Viral coefficientOn average, for each message they hear, how many people does someone tell?

1Wednesday, December 8, 2010

------------------------------------------------------Get your free private email at http://www.hotmail.com------------------------------------------------------

!Wednesday, December 8, 2010

v ≠ 1, pt = δp0 (1 – vt+1) / (1 – v) + p0

http://robert.zubek.net/blog/2008/01/30/viral-coefficient-calculation/Wednesday, December 8, 2010

How well are messages being amplified?

50% 75% 100% 50% = 4

2 3 1 = 6

Downstream reach (aggregate followers)

Amplification chance(how relevant the content

is + what the recipient thinks of the sender)

1 = 3.5

Repurposing(spread to othercommunities)

Amplification(virality andmessage spread)

Conversions

Visits

Shopping cart

Payment options

Seed (startingcommunity)

Reach (impressions)

Viral message spread

Conversions

Visits

Shopping cart

Payment options

Reach (impressions) Emphasis on

getting viral ratio above 1(Retweeting, Fan, Email

forward, Reddit upvote, other

loops)

Megablogger proponents

Conversions

Visits

Shopping cart

Payment options

Seed (startingcommunity)

Reach (impressions)

Emphasis on convincing highly-followed, highly

acted-upon seed group to spread

the word.

A call to action

Conversions

Visits

Shopping cart

Payment options

Reach (impressions)

Emphasis on maximizing

impression-to-click ratio within the community

Goal attainment: funnel

700,000s

x $100x $20

20Seed ratio:

35,000:1

Followers

Visitors:0.23%

Conversions:1.95%

Revenues:Average: $39.54Median: $20

Total: $1,005

Amplification: 2.9%

2 Repurposing

How performable are we?Health of the infrastructure (uptime, latency)

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16

>4s or error

Conversions

$10,000

Revenue (total sales)

What do visitors think of us?Both on the site (VoC) and in the world (community monitoring)

4&5Wednesday, December 8, 2010

How engaged are users?Time since last visit, as a histogram, not an average, compared to baselines.

Days since last visit

01 2 3 4 5 6 7 8 9

Days since last engagement

January February

Disengaged(>10 days)

Core goals driving the businessPick the 3-4 tasks you’ve set for users. Always know what these are and why they correlate to business growth.

Revenue Lemonade( ) x Units sold + tips

Lemon + cups + sugar( ) x Units made

Markers + cardboard( ) x Signs made

Staffing( ) x Hours worked

Profit!

(Remember the model?)

Extended funnel abandonmentNot just on the site: Email open rate

In the last build, what moved us towards & away from goals?Requires knowing the goals and the key features added

Cost per visitorOperations costs per site visitor

(this had better be greater than the revenues and conversion rate.)

Peak-to-average ratioHow much busier are you at busy times?

What premium do you pay for that surplus?

Can you decommission well?

http://community.microsoftadvertising.comWednesday, December 8, 2010

http://www.flickr.com/photos/dougww/833127501/Wednesday, December 8, 2010

Message reach

Infrastructure health

Market sentiment

User engagement

Business model

Sustainability

Viral coefficient

Amplification

Performability

Customer research

Community sentiment

Time since last visit

Core goals

Extended funnel abandonment

Movement towards goals

Cost per engaged visitor

Peak to average ratio

12thingsto

Thanks!alistair@bitcurrent.com

@acrollwww.yearonelabs.com

www.rpmmtl.com

www.watchingwebsites.com

www.bitcurrent.com

www.human20.com

lean analytics for startups - leweb2010

header 200ms

signs madestaffing

2010known

hours workedprofit

2010costrevenue

engaged visitorpeak

average

time

Technology

lean startups & marketing automation

lean startups - red cide

lean marketing for startups by andrew

nick boucart lean startups

lean fluffy startups

introduction to running lean startups

analytics for startups

lean pricing startups

a/b testing for lean startups

lean startups & micro ideas

lean startups

lean startups steve blank eric ries

marketing for lean startups

agile content: lessons from lean startups

metrics for lean startups

lean internationalization of high-tech startups

big data startups - top visualization and data analytics...

google analytics for startups

lean branding for startups

lean startups and strategic management