growth models: the science of converting users …...3. weekly experimentation 2. growth models...

with Scott Tousley

Growth Models: The Science of Converting Users into Customers

@sjtousley

● Freemium user acquisition at HubSpot

● Co-host of The Growth TL;DR Podcast

● Growth advisor to startups

● Nomadic / location independent

Hi! I’m Scott Tousley

@sjtousley

Macro Growth Model

A spreadsheet that helps you quantify and predict how your business

holistically grows.

What are growth models?

Micro Growth Model

A micro growth model is a way to predict experiment results and their

impact on “zoom out metrics”.

@sjtousley

Process > Model > Experiments

GROWTH PROCESS

GROWTH MODEL

EXPERIMENTS

1 Growth Process

@sjtousley

Annual

W12W10

W11W12W13

W4W3 W2 W1

@sjtousley

Q1“zoom out”

W12W10

W11W12W13

W4W3 W2 W1

Q2“zoom out”

Q3“zoom out”

Q4“zoom out”

North Star Metric

@sjtousley

Weekly Active Team

W12W10

W11W12W13

W4W3 W2 W1

User Activation Rate

Team Activation Rate

Week 12 Retention New Users

Acquired

@sjtousley

Weekly Active Team

W12W10

W11W12W13

W4W3 W2 W1

Acquired

Experiment

ExperimentExperiment

Experiment

@sjtousley

Weekly Active Team

W12W10

W11W12W13

W4W3 W2 W1

Acquired

Experiment

@sjtousley

The Growth Process

1. Annual North Star Metric

2. Quarterly ”Zoom Out” Metrics

3. Weekly Experimentation

2. Growth Models

@sjtousley

Process > Model > Experiments

GROWTH PROCESS

GROWTH MODEL

EXPERIMENTS

@sjtousley

Q1“zoom out”

W12W10

W11W12W13

W4W3 W2 W1

Q2“zoom out”

Q3“zoom out”

Q4“zoom out”

North Star Metric

@sjtousley

Q1“zoom out”

W12W10

W11W12W13

W4W3 W2 W1

Q2“zoom out”

Q3“zoom out”

Q4“zoom out”

North Star Metric

@sjtousley

Pirate Metrics

A_______

R_______

R______

R_____

A________

@sjtousley

Pirate Metrics

Activation

Retention

Referral

Revenue

Acquisition

@sjtousley

Pirate Metrics

Activation

Retention

Referral

Revenue

Acquisition …. people sign up for your product/service

…. people experience “aha moment”

…. people continually come back to you

…. people tell your friends about you

…. people pay you money

@sjtousley

Case Study: Facebook

Activation

Retention

Referral

Revenue

Acquisition …. Create Facebook account

…. Add 7 friends in 10 days

…. Sign in once per day

…. Email Facebook invite to a friend

…. Browse news feed, see ads, FB makes $$$

@sjtousley

Activation

Retention

Referral

Revenue

Daily Active User (DAU)

@sjtousley

Activation

Retention

Referral

Revenue

North Star Metric

@sjtousley

Activation

Retention

Referral

Revenue

Acquisition …. Create Spotify account

…. Listen to 100 songs in 5 days

…. Listen for at least 1 hour per week

…. Share song with your friend

…. Upgrade to Spotify Premium for $10/month

Case Study: Spotify

@sjtousley

Activation

Retention

Referral

Revenue

Weekly Active User (WAU)

Case Study: Spotify

@sjtousley

Activation

Retention

Referral

Revenue

North Star Metric

Case Study: Spotify

@sjtousley

Tomorrow’s workshop!

Activation

Retention

Referral

Revenue

Acquisition …. ______________________

…. ______________________

North Star Metric

@sjtousley

Activation

Retention

Referral

Revenue

Acquisition …. Create free HubSpot CRM account

…. 2 people view 1 contact record

…. 2 people view 1 contact record per week

…. Tag coworker on contact record (via @dannos)

…. Upgrade to paid HubSpot products

Case Study: HubSpot CRM

@sjtousley

Activation

Retention

Referral

Revenue

Weekly Active Team (WAT)

@sjtousley

Activation

Retention

Referral

Revenue

North Star Metric

@sjtousley

Weekly Active Team = New Teams Activated + Existing Teams Retained - Existing Teams Lost

New Teams Activated = New Teams Acquired * Team Activation Rate

New Teams Acquired = Teams from SEO + Teams from Paid + Teams from Affiliate + ……..

Teams from Paid = Number of impressions * ad conversion rate * landing page conversion rate

Landing page conversion rate = Copywriting + Visual Design + User Experience

North Star

Metric

Zoom out metrics

Zoom in experiments

@sjtousley

Case Study: HubSpot CRMNorth Star

Metric

Zoom out metrics

@sjtousley

Metric

Zoom out metrics

??????

@sjtousley

Metric

Zoom out metrics

@sjtousley

Metric

Zoom out metrics

@sjtousley

??? = 10,000 * 30%

Metric

Zoom out metrics

@sjtousley

3,000 = 10,000 * 30%

Metric

Zoom out metrics

@sjtousley

Metric

Zoom out metrics

@sjtousley

10,000 = 5,000 + ???? + 1,500

Metric

Zoom out metrics

@sjtousley

10,000 = 5,000 + 3,500 + 1,500

Metric

Zoom out metrics

@sjtousley

North Star

Metric

Zoom out metrics

Zoom in experiments

@sjtousley

Landing page conversion rate = Copywriting + Visual Design + User Experience

North Star

Metric

Zoom out metrics

Zoom in experiments

@sjtousley

● Apply for the Reforge Growth Series course by Brian Balfour

● “Indispensable Growth Frameworks from My Years at Facebook, Twitter and Wealthfront” by Andy Johns

● “Why Retention is the Silent Killer” by Brian Balfour

● “6 Growth Experiments from HubSpot to Improve Retention” by Dan Wolchonok

● “The Evolution of Acquisition Channels” by James Currier

● “The Law of Shitty Clickthroughs” by Andrew Chen

● “Why Activation is the Most Important Pirate Metric” by Ty Magnin

Further Reading on Growth Models & Strategy

3. Experiments

@sjtousley

Weekly Active Team

W12W10

W11W12W13

W4W3 W2 W1

Acquired

Experiment

@sjtousley

Weekly Active Team

W12W10

W11W12W13

W4W3 W2 W1

Acquired

Experiment

8 in 10 experiments you run will fail.

Failed Experiments = Learning

Successful Experiments = Learning

Inconclusive = Learning

More experiments = Faster learning

Faster learning = Better predictions

Better predictions = More productivity

More productivity = Faster growth

@sjtousley

More Experiments = Faster Growth

How you can experiment and learn ….

● A/B test ● Multivariate tests● URL redirects ● Smoke tests ● Heatmaps/clickmaps● Qualitative surveys● Quantitative surveys● Conversations after experiments!

Where you can experiment and learn….

● Blog posts ● Homepages ● Pricing pages● Shopping carts (ecommerce) ● Product pages ● Paid ads for direct sales● Paid ads for leads● Email (one off)● Email (series of automated emails) ● Tweets, Facebook posts, etc● And many more….

@sjtousley

The experimental framework

Identify

Ideate

PrioritizeDocument

Analyze

Share1. Identify the metric

2. Ideate list of experiment ideas to improve metric

3. Prioritize ideas with P.I.E.

4. Write experiment document & turn experiment live

5. Analyze results in experiment document

6. Share results with team

@sjtousley

Experiment Example: HubSpot CRM

Problem: Inside the product, users have one option if they want to talk to sales -- book a meeting in the future.

Question: However, what if users want to talk to sales now?

Disclaimer

All data is fake! However, all learnings are very real.

@sjtousley

Problem: Today’s UX

@sjtousley

The experimental framework

Identify

Ideate

PrioritizeDocument

Analyze

@sjtousley

PQL (Product Qualified Lead) Revenue = Total PQLs * Close Rate * Avg. Sales Price

Close Rate = Copywriting + Segmentation + Design + Response Time

Zoom in experiments

Identify the metric

@sjtousley

Ideate list of ideas

Identify

Ideate

PrioritizeDocument

Analyze

@sjtousley

1. Identify the metric

Ideate list of ideas

@sjtousley

Prioritize ideas

Identify

Ideate

PrioritizeDocument

Analyze

@sjtousley

Potential Impact Ease

What is the potential for improvement? What is the ceiling and are we close to it?

1 = little room to improve

5 = much room to improve

How big of impact do you predict this to have on your target metric?

1 = little impact on target metric

5 = massive impact on target metric

How easy will this be to implement? Consider both technical and political.

1 = difficult

5 = easy

Prioritize ideas

@sjtousley

Prioritize ideas

@sjtousley

Zoom in

experiment!

Prioritize ideas

@sjtousley

Write document

Identify

Ideate

PrioritizeDocument

Analyze

@sjtousley

Experiment Design: Proposed UX

@sjtousley

Analyze results

Identify

Ideate

PrioritizeDocument

Analyze

@sjtousley

Analyze resultsTotal PQLs: 219 Total Conversations: 37

CONTROL

@sjtousley

Analyze resultsTotal PQLs: 160 Total Conversations: 74

VARIANT

@sjtousley

Link to data - https://docs.google.com/spreadsheets/d/1Tb8SRferibKjBZ8T-ORE569NN5VTLr530dO3sKK0VY0/edit#gid=0

@sjtousley

ResultsTotal PQLs: 160 Total Conversations: 74

We’ve nearly tripled the amount of sales conversations.

17% → 46%

@sjtousley

Share results

Identify

Ideate

PrioritizeDocument

Analyze

@sjtousley

Share results

@sjtousley

The tools you need to run experiments

● Google Spreadsheets to analyze North Star and “zoom out” metrics growth

● Airtable or Trello as project management tool (experiment backlog)

● Google Presentations to visualize your experiments

● Google Optimize to run A/B tests on your website

● Google Analytics to analyze results on your website

● Atlassian’s Wiki as central place to share “What did we learn this week?”

● Statistical significance calculator to double check results

● HubSpot (or similar tool) to run email A/B tests with marketing automation

● Hotjar for heatmaps, clickmaps, and visitor recordings

● Marketing tools like SEMRush, Buzzsumo, Hunter.io, Ahrefs, etc

@sjtousley

Identify

Ideate

PrioritizeDocument

Analyze

growth models: the science of converting users …...3. weekly experimentation 2. growth models...

Documents

optimal design of experiments for the quadratic...

experiments and numerical models for eigenfrequency

random growth models

pilot plant experiments and modeling of caco3 growth

probabilistic topic models - uci cognitive science...

models and experiments of active bodies interacting with

micro-scale experiments and models for composite materials

matchstick models and other science experiments

growth models

coalescence of liquid drops: different models vs ...

experiments testing models of mind-matter interaction ·...

g1 mechanical models experiments

experiments with different models of statistcial machine...

growth experiments: lessons from a failed attempt

section 6.8 exponential growth and decay models; logistic...

building models for complex design of experiments

independent experiments on seed germination or seedling...

predictive models and experiments for high-velocity and

a 3u cubesat platform for plant growth experiments

lmdme: linear models on designed multivariate experiments in...