using aarrr metrics to guide product decisions @ knowledgevision

57
USING AARRR METRICS TO GUIDE PRODUCT DECISIONS @KNOWLEDGEVISION Matt Kaplan & Alex Kieft Wireless Access: Connect to Cambridge “st0126”

Upload: chen

Post on 25-Feb-2016

45 views

Category:

Documents


0 download

DESCRIPTION

Wireless Access: Connect to Cambridge “st0126”. Using AARRR METRICS to GUIDE Product Decisions @ knowledgevision. Matt Kaplan & Alex Kieft. Lean Startup Circle Boston, Jan 2012. Wireless Access: Connect to Cambridge “st0126” Feb 17: Lean Startup Machine 10% off with code: “ LeanCircle ” - PowerPoint PPT Presentation

TRANSCRIPT

Page 1: Using AARRR METRICS to GUIDE Product Decisions @ knowledgevision

USING AARRR METRICS TO GUIDE PRODUCT DECISIONS

@KNOWLEDGEVISION

Matt Kaplan & Alex Kieft

Wireless Access: Connect to Cambridge “st0126”

Page 2: Using AARRR METRICS to GUIDE Product Decisions @ knowledgevision

Lean Startup Circle Boston, Jan 2012

Wireless Access: Connect to Cambridge “st0126”

Feb 17: Lean Startup Machine10% off with code: “LeanCircle”

Register Now:http://theleanstartupmachine.com/

events/boston-february-17/

Page 3: Using AARRR METRICS to GUIDE Product Decisions @ knowledgevision

What is AARRR?

Acquisition Users come to the site from various channelsActivation Users enjoy 1st happy visitRetention Users come back, visit multiple timesReferral Users like product enough to refer othersRevenue Users conduct some monetization behavior

Dave McClure, creator of “Pirate Metrics”@davemcclurehttp://500hats.typepad.com

Page 4: Using AARRR METRICS to GUIDE Product Decisions @ knowledgevision

Why is AARRR important?

Creates a Common Language

Page 5: Using AARRR METRICS to GUIDE Product Decisions @ knowledgevision

Why is AARRR important?

Creates a Common LanguageProvides

Meaningful Feedback

Page 6: Using AARRR METRICS to GUIDE Product Decisions @ knowledgevision

Why is AARRR important?

Creates a Common LanguageProvides

Meaningful Feedback

Guides Decision Making

Page 7: Using AARRR METRICS to GUIDE Product Decisions @ knowledgevision

Why is AARRR important?

Creates a Common LanguageProvides

Meaningful Feedback

Guides Decision Making

ResolvesConflict

Page 8: Using AARRR METRICS to GUIDE Product Decisions @ knowledgevision

About KnowledgeVision

Page 9: Using AARRR METRICS to GUIDE Product Decisions @ knowledgevision

New Product Initiative Objectives:

Easy enough for any user Using only webcam and PowerPoint Web only, no software to download

MVP Register/Login Upload PowerPoint Record presentation Save presentation Email link

Page 10: Using AARRR METRICS to GUIDE Product Decisions @ knowledgevision

Knovio – Born 6/30/11, 11:55pm

Page 11: Using AARRR METRICS to GUIDE Product Decisions @ knowledgevision

Knovio – Upload Slides

Page 12: Using AARRR METRICS to GUIDE Product Decisions @ knowledgevision

Knovio – Record Video

Page 13: Using AARRR METRICS to GUIDE Product Decisions @ knowledgevision

Knovio Presentation

Page 14: Using AARRR METRICS to GUIDE Product Decisions @ knowledgevision

Launch Strategy Private beta program Friends & family

seeding Leverage educators

and students for testing

Outreach to influential bloggers in Education

Get email with activation code

Sign-up

ActivateAccount

1

23

Check out

Page 15: Using AARRR METRICS to GUIDE Product Decisions @ knowledgevision

New User Growth in First Month

6/30/1

17/2

/117/4

/117/6

/117/8

/11

7/10/1

17/1

2/117/1

4/117/1

6/117/1

8/117/2

0/117/2

2/117/2

4/117/2

6/117/2

8/117/3

0/11

8/1/11

0

20

40

60

80

100

120

140

0

100

200

300

400

500

600

700

800

900

1000

Knovio User Growth

New UsersTotal Users

Date

Use

rs

Page 16: Using AARRR METRICS to GUIDE Product Decisions @ knowledgevision

Knovio Traffic

Page 17: Using AARRR METRICS to GUIDE Product Decisions @ knowledgevision

Global Interest in Knovio

Page 18: Using AARRR METRICS to GUIDE Product Decisions @ knowledgevision

From zero to 29,400 results in 30 days

Page 19: Using AARRR METRICS to GUIDE Product Decisions @ knowledgevision

“…one of the best Web 2.0 applications of the year.”

Page 20: Using AARRR METRICS to GUIDE Product Decisions @ knowledgevision

Acquisition Metrics

Stage Metric

Snapshot on July 7

Acquisition Visits Knovio homepage 100%Activation

Signs up for Knovio private beta 40%

Activation

Activates Knovio account (of signups) 58%

Activation

Creates at least 1 Knovio (of signups) 11%

Page 21: Using AARRR METRICS to GUIDE Product Decisions @ knowledgevision

Acquisition Metrics

Stage Metric

Snapshot on July 7

Snapshot on July

31Acquisition Visits Knovio homepage 100% 100%

Activation

Signs up for Knovio private beta 40% 20%

Activation

Activates Knovio account (of signups) 58% 59%

Activation

Creates at least 1 Knovio (of signups) 11% 12%

Initial sign-ups were biased

by friends & family

Page 22: Using AARRR METRICS to GUIDE Product Decisions @ knowledgevision

Other Acquisition StrategiesInitial

Friends & Family

Blogger Outreac

h Pay per click

Organic

Page 23: Using AARRR METRICS to GUIDE Product Decisions @ knowledgevision

Acquisition Metrics

Stage Metric Jul AugAcquisition Visits Knovio homepage 100% 100%

Activation

Signs up for Knovio private beta 20% 6%

Activation

Activates Knovio account (of signups) 59% 63%

Activation

Creates at least 1 Knovio (of signups) 12% 19%

New acquisition strategies were not producing positive

resultsSome positive

signs of improvement

Page 24: Using AARRR METRICS to GUIDE Product Decisions @ knowledgevision

A/B Testing – Swap buttons14%

Check out

Page 25: Using AARRR METRICS to GUIDE Product Decisions @ knowledgevision

A/B Testing – One button26%

Page 26: Using AARRR METRICS to GUIDE Product Decisions @ knowledgevision

A/B Testing – New Concept 27%

Page 27: Using AARRR METRICS to GUIDE Product Decisions @ knowledgevision

Stimulate User Activity

Fun, camp-themed email newsletter

Weekly, personalized activities

Specific recommendations and targets

Testing limits and game mechanicsCheck out

35%Open Rate

Page 28: Using AARRR METRICS to GUIDE Product Decisions @ knowledgevision

Acquisition Metrics

Metric Jul Aug Sep OctVisits Knovio homepage 100% 100% 100% 100%

Signs up for Knovio 20% 6% 7% 18%

Activates Knovio account 59% 63% 68% 73%

Creates at least 1 Knovio 12% 19% 26% 36%

Improved activation

Page 29: Using AARRR METRICS to GUIDE Product Decisions @ knowledgevision

Product Improvements Most requested features

Delete presentation Embed code for sharing Audio-only presentations Shorter URLs Re-recording slides Editing presentations Check out

Page 30: Using AARRR METRICS to GUIDE Product Decisions @ knowledgevision

Involved users in feature design

Page 31: Using AARRR METRICS to GUIDE Product Decisions @ knowledgevision

Public Beta – Nov 15, 2011 Redesigned homepage Removed extra step in

registration process Targeted business user Improved messaging Aligned with core

product 200+ Bloggers

Page 32: Using AARRR METRICS to GUIDE Product Decisions @ knowledgevision

Acquisition Metrics

Metric Jul Aug Sep Oct Nov DecVisits Knovio homepage

100%

100%

100%

100%

100%

100%

Signs up for Knovio 20% 6% 7% 18% 16% 18%

Activates Knovio account 59% 63% 68% 73% 100

%100%

Creates at least 1 Knovio 12% 19% 27% 36% 37% 44%

Much better activation metrics across the board

Page 33: Using AARRR METRICS to GUIDE Product Decisions @ knowledgevision

Knovio User Growth

Jul Aug Sep Oct Nov Dec0

500

1000

1500

2000

2500

3000

3500

Use

rs

Avoid “feel-good” vanity Metrics

Page 34: Using AARRR METRICS to GUIDE Product Decisions @ knowledgevision

Initial Retention Metrics

Metric Jul Aug SepCreates 2 or more presentations 5% 7% 10%Returns to create presentation (following months) 5% 6% 8%

Logs in 3 times or more ? ? ?

Hypothesis: If users could get analytics and feedback on presentations, they would use it more

Page 35: Using AARRR METRICS to GUIDE Product Decisions @ knowledgevision

New Feedback Features Email alerts upon viewing Built in feedback

“A user from Concord, MA just watched your Knovio presentation”

Mary M. said “This is a great presentation, please make more!”

Page 36: Using AARRR METRICS to GUIDE Product Decisions @ knowledgevision

Retention Metrics

Metric JulAug

Sep Oct

Nov

Dec

Creates 2 or more presentations 5% 7% 10

%17%

12%

15%

Returns to create presentation (following months)

5% 6% 8% 7% 14%

TBD

Logs in 3 times or more 35%

55%

Rolled out email alerts

upon viewingBegan tracking

login history

Page 37: Using AARRR METRICS to GUIDE Product Decisions @ knowledgevision

Cohort Report for Retention Analysis

Jul Aug Sep Oct Nov Dec0.0%

5.0%

10.0%

15.0%

20.0%

25.0%

30.0%

35.0%

40.0%

45.0%

Born in JulBorn in AugBorn in SepBorn in OctBorn in NovBorn in Dec

Activ

e U

ser

%

Page 38: Using AARRR METRICS to GUIDE Product Decisions @ knowledgevision

Cohort Report – A better view

Jul Aug Sep Oct Nov Dec0.0%

5.0%

10.0%

15.0%

20.0%

25.0%

30.0%

35.0%

40.0%

45.0%

Month BornMonth +1Month +2Month +3Month +4Month +5

Page 39: Using AARRR METRICS to GUIDE Product Decisions @ knowledgevision

Knovio User Types

44%

34%

22%Note: Beware of

otherfactors effecting

metrics

Page 40: Using AARRR METRICS to GUIDE Product Decisions @ knowledgevision

Hypothesis: Viral features will fuel growth

User creates 1 Knovio

presentation

Emails presentation

Posts presentation

to FB, Twitter, LinkedIn

Invites friend to

use Knovio

20 viewers x 2% = .4

50 viewers x 1% = .5

5 invites x 20% = 1

Viral Coefficient = 1.9David SkokMatrix PartnersForEntrepreneurs.com

Page 41: Using AARRR METRICS to GUIDE Product Decisions @ knowledgevision

Referrals - Sharing built into app

Link Embed Email Social Media

Page 42: Using AARRR METRICS to GUIDE Product Decisions @ knowledgevision

Email deliverability

Check out

57% open rate

94% deliverability

Page 43: Using AARRR METRICS to GUIDE Product Decisions @ knowledgevision

Metrics for Sharing Options

Get Embed

Get LinkEmail

Check out

Page 44: Using AARRR METRICS to GUIDE Product Decisions @ knowledgevision

SelfFeedback

PrivateSharing &Feedback

PublicSharing

& Feedback

Referrals - Sharing Hypothesis

Requires Registration

Page 45: Using AARRR METRICS to GUIDE Product Decisions @ knowledgevision

New Feature: “Spaces”

Page 46: Using AARRR METRICS to GUIDE Product Decisions @ knowledgevision

Knovio “Spaces”

Page 47: Using AARRR METRICS to GUIDE Product Decisions @ knowledgevision

Spaces – Invite Users

Page 48: Using AARRR METRICS to GUIDE Product Decisions @ knowledgevision

Metrics for Sharing Options

Get Embed

Get LinkEmail

Check out

Spaces

Page 49: Using AARRR METRICS to GUIDE Product Decisions @ knowledgevision

Idea: Entice users to try Spaces

Page 50: Using AARRR METRICS to GUIDE Product Decisions @ knowledgevision

Referral Metrics

Metric Jul Aug Sep Oct Nov DecShare via Email 7% 11% 14% 23% 16% 21%Invite via Spaces 1.8% 2.9% 2.4%

Better, but not exactly what we were hoping for

Good improvement but unpredictable

Page 51: Using AARRR METRICS to GUIDE Product Decisions @ knowledgevision

Revenue Hypothesis

KVStudioPower Tools for

rich content producers

Lead generation

Page 52: Using AARRR METRICS to GUIDE Product Decisions @ knowledgevision

Revenue Metrics

Metric Jul Aug Sep Oct Nov DecLeads for KVStudio .2% .5% 0% .6% 1.4%

Ugh

Page 53: Using AARRR METRICS to GUIDE Product Decisions @ knowledgevision

AARRR Lessons Learned Create your AARRR dashboard before

MVP Build metrics capture into product Measure only what you think you’ll need Don’t change too many variables at once Know who your target is and only listen

to them Get entire company on board (not easy!)

Page 54: Using AARRR METRICS to GUIDE Product Decisions @ knowledgevision

Too many metrics

Page 55: Using AARRR METRICS to GUIDE Product Decisions @ knowledgevision

Tools for capturing metrics/feedback

Prefinery – Private beta management Mixpanel – Tracking user events Unbounce – A/B testing Wufoo – User feedback MailChimp – Email engagement SendGrid – Transactional email Amazon SimpleDB – Error logs

Page 57: Using AARRR METRICS to GUIDE Product Decisions @ knowledgevision

Lean Startup Circle Boston, Jan 2012

Wireless Access: Connect to Cambridge “st0126”

Feb 17: Lean Startup Machine10% off with code: “LeanCircle”

Register Now:http://theleanstartupmachine.com/

events/boston-february-17/