qual vs. quant: using data to build better products

Qual vs QuantUsing Better Data to Build Better Products

Laura Kleinlaura@usersknow.com

@lauraklein

Who’s Tired Of Hearing About Data?*

*be honest

A/B Testing

A/B Testing

Funnel Analysis

A/B Testing

Cohorts

Funnel Analysis

A/B Testing

Cohorts

Funnel Analysis

Blah

A/B Testing

Cohorts

Funnel Analysis

Blah

Blah

A/B Testing

Cohorts

Funnel Analysis

Blah

Blah

Blah

Sorry. I’m Going To Talk About Data Some More.

Better Data==

Better Products

Well…Sometimes

What Data Can’t DoReplace Design

Replace Listening to Users

Tell Us What We Should Be Building

What Data Can DoInform Design

Inform Research

Tell Us What We Should Be Investigating

How About An Example?

A Funnel

A Checkout Funnel

Create Account

Select Plan

Payment Info

Address

ConfirmThanks & Up Sell

A Leaky Checkout Funnel

100%

Create Account

Select Plan

Payment Info

Address

ConfirmThanks & Up Sell

-20%

-10%

-50%

-33%

-25%== ???

What That Means to You

100 Buyers

Create Account

Select Plan

Payment Info

Address

ConfirmThanks & Up Sell

-20 Buyers

-8 Buyers

-36 Buyers

-12 Buyers

-6 Buyers== 18 Buyers

100 To 18????How Do We Fix This?

Identifying a Friction Point

100 Buyers

Create Account

Select Plan

Payment Info

Address

ConfirmThanks & Up Sell

-20 Buyers

-8 Buyers

-36 Buyers

-12 Buyers

-6 Buyers== 18 Buyers

Stop It

Quantitative Data Qualitative Data

Understanding the Friction Point

Quantitative Data

WHAT

Qualitative Data

WHY

How Can You Use Them Together?

Step 1: Identify the Biggest Problem

Step 1: Identify the Biggest Problem

Use Funnel Metrics

The Biggest Problem

100 Buyers

Create Account

Select Plan

Payment Info

Address

ConfirmThanks & Up Sell

-20 Buyers

-8 Buyers

-36 Buyers

-12 Buyers

-6 Buyers== 18 Buyers

Step 1: Identify the Biggest Problem

Use Funnel Metrics Quant

Step 1: Identify the Biggest Problem

Use Funnel Metrics

Step 2: Understand Why

Step 1: Identify the Biggest Problem

Use Funnel Metrics

Step 2: Understand Why

Observational Usability

Step 1: Identify the Biggest Problem

Use Funnel Metrics

Step 2: Understand Why

Observational Usability Qual

Problem Possibilities

Create Account

Select Plan

Payment Info

Address

ConfirmThanks & Up Sell

-36 People• Didn’t have payment info handy• Confused by specific question• Looked for promo code

Step 1: Identify the Biggest Problem

Use Funnel Metrics

Step 2: Understand Why

Observational Usability

Step 3: Propose Solutions

Step 1: Identify the Biggest Problem

Use Funnel Metrics

Step 2: Understand Why

Observational Usability

Step 3: Propose Solutions

Create Solution Hypotheses

Solution Hypothesis

Problem

People looked for a promo code.

Solution Hypothesis

Possible Solution

Remove promo code entirely.

Solution Hypothesis

What Do We Think Would Happen?

An increase in conversion.

Solution Hypothesis

What Might Go Wrong?

A decrease in acquisition or revenue.

Step 4: Learn & Iterate*

*The last step is always learn & iterate!

Quant & Qual

Learn…

How Would We Know We Were Right?

A/B test of the checkout flow with and without the promo code.

What Really Happened?

Create Account

Select Plan

Payment Info

Address

ConfirmThanks & Up Sell

Why Did It Happen?

Qual!

…And Iterate

Another Possible Solution

Create custom landing pages rather than promo codes.

Step 1: Identify the Biggest Problem

Use Funnel Metrics

Step 2: Understand Why

Observational Usability

Step 3: Propose Solutions

Create Solution Hypotheses

Step 4: Learn & Iterate!

Or…

Step 1: Identify the Biggest Problem

Step 1: Identify the Biggest Problem

Interview Current UsersQual

Step 1: Identify the Biggest Problem

Interview Current Users

Step 2: Understand What

Step 1: Identify the Biggest Problem

Interview Current Users

Step 2: Understand What

Study Relevant MetricsQuant

But Why?

Step 1: Identify the Biggest Problem

Interview Current Users

Step 2: Understand What

Study Relevant Metrics

Step 3: Propose Solutions

Create Solution Hypotheses

Step 1: Identify the Biggest Problem

Look at Your Sales Pipeline

Step 2: Understand Why

Interview People Who Said No

Step 3: Propose Solutions

Create Solution Hypotheses

Step 4: Learn & Iterate*

*The last step is always learn & iterate!

Once Again…

Quantitative Data

WHAT

Qualitative Data

WHY

Better Data ==

Better Products

Questions?

@laurakleinlaura@usersknow.com