tripling your conversion rates with a/b testing

Tripling Your Conversion Rates With A/B TestingLars Lofgren

@larslofgren

2X and 3X in 9-12 months

KISSmetrics homepage conversion to signups.

Nov 2013 = 2.85% Aug 2014 = 8.71%

• Why not just launch stuff? Why A/B test?

• The Growth March

• Prepping for Your March

• 7 Rules of A/B Testing

We’ll cover...

But Lars! I already know how to increase

conversions!

My win-rates at KISSmetrics after 65 A/B tests.N

umbe

r of W

ins

0

10

20

30

40

Confirmed Loser Marginal Confirmed Winner

At best, 28% of your tests will win.

If most of our tests fail…we can’t depend on any

single test.

The Growth March

Growth is not a project, it’s a relentless grind.

A/B testing is about playing the long-term.

You’re in a quantity game. More tests = more wins.

Prepping for Your March

Get your data in order.

Need to tie test segments through entire funnel.

Get your team in place.

If your tests involve multiple teams, you won’t get far.

The Bare-Bones Growth Team

• PM, background in marketing or product

• Designer that loves iteration and results

• Front-end engineer, full-stack if you can spare

Get your levers prioritized.

Don’t waste tests on assets that impact only a small

number of people.

First, fix any broken steps in your funnel.

Start at the top of your funnel and work down.

Which of your assets drive the most conversions?

Get research for testing.

Bad ideas for testing.

You won’t find majorwins from random“growth hacks”.

You need a stream of insights from your

customers.

1. Interviews with customers from last 30 days.

2. Surveys for people that abandoned funnels.

3. Qualaroo pop-ups at funnel bottlenecks.

4. Heatmaps on struggling pages.

5. Usability tests in signup and lead flows.

Great places for testing research.

1. Get your data in order.

2. Get your team in place.

3. Get your levers prioritized.

4. Get research for testing.

The Growth March Prep

7 Rules of A/B Testing

A/B tests aren’t for picking winners. They’re for

avoiding losers.

False positives will destroy your gains.

Cycle through as many tests as possible while only

launching confirmed winners.

1. Above all else, the control stands.

2. Get 2000+ people through the test within 30 days.

3. Wait at least 1 week before checking data.

4. Only launch variants at 99% statistical significance.

Same test, 4 weeks later.

5. If the test drops below 10%, ditch it.

6. If no new winner after a month, ditch variant.

7. Get the next test ready while you wait.

1. Above all else, the control stands.

2. Get 2000+ people through the test within 30 days.

3. Wait at least one week before checking data.

4. Only makes changes at 99% certainty.

5. If test drops below 10%, ditch it.

6. If no winner after 1 month, ditch variant.

7. Build your next test while waiting on data.

7 Rules of the Growth March

Questions?Lars Lofgren@larslofgren