designing work

DesigningExperimentan

This is a brief guide to designing experiments

This is a brief guide to designing experimentsWARNIN

G!

Be sure to fill in all blanks before proceeding to the next step. Skipping this step will prevent you from getting the most out of this guide.

Now, with loss aversion in mind, let’s proceed to the next step...

This is a brief guide to designing experimentsWARNIN

G!

Be sure to fill in all blanks before proceeding to the next step. Skipping this step will prevent you from getting the most out of this guide.

Identify Key Behavior1

What is the key behavior you want your users to

do?

Identify Key Behavior1

e.g. sign up for workshop

Identify Key Behavior1

Key Behavior =Measured Variable (what you will measure to see the effect of your test)

?EXPERT TIPAsk around, does

everyone involved with your project agree that this is the most important behavior to measure?

Identify Behavioral Barriers 2

Identify Behavioral Barriers 2

What are the barriers preventing people from

doing the key behavior that you want them to do?

not enough time

read e-mail and forget about itput off until later

(list as many as you can think of)

Choose a Barrier3

A B C D

Choose a Barrier3

Choose the behavioral barrier (1) that you think contributes most to people not doing the key behavior, and explain it in the following format:

A B C D

Choose a Barrier3

The reason people are not currently partaking in

adopting negative keywords is because they put it off until later

key behavior

behavioral barrier

A B C D

4 Removing Barriers

Think about behavioral economic principles. What are some ways that you can remove this barrier? (list as many as you can*)

4 Removing Barriers

Precommitment: Put it on their calendar for them

In-your way instructions: Have a pop-up when they log into account

Urgency: Provide a deadline for the action to be completed

4 Removing Barriers

4 Removing Barriers*If you don’t

know principles, check out our workbooks

Choose the approach that you think is most likely to remove the barrier.

What is the behavioral economic principle that you are using?

5Choose a

Theory

By using urgency, we can increase the number of

people who adopt negative

keywords.

5Choose a

Theory

If we do this...then this will happen...

because...

If we add a sense of urgency, then people will be more likely to take action to add

negative keywords, because urgency gives people a clear timeline in which they must

complete the action in order to get the incentive

Write the Hypothesis

Creating Conditions(Control & Test)

How would you test this? Let’s start off with just two conditions (your most basic A/B test). Design your test below. What will your each group of participant see?

• Write out the exact text they will see?• Will you use images? If so, add the exact images they will see?

Control Test

Must have at least two conditions - control condition and test condition The control condition must be

exactly the same as the test condition, except for the ONE key difference you are testing.

Creating Conditions

Control Test

You must have a control

Make your ad more relevant. Adopt keywords and receive $50 in AdWords Credit

Make your ad more relevant. Adopt keywords in the next 7 days and receive $50 in AdWords Credit

Control Test

•The difference between the control and the test is the one thing you are testing

•Urgency: “In the next 7 days”•Restricting to only one difference between any two condition allows you to learn something from results.

•Imagine that you include 2 difference in the test (e.g., urgency and amount of incentive), and let’s imagine that the test works twice as well as the control. Why did it happen? You don’t know - it could

have been urgency it could have been the amount of the incentive, or a combination of both)

8Midterm Review

8Midterm Review

Do I have a control condition?

Do I have only one difference

between the two conditions?

Checklist

8Midterm Review

Given the information I provided,Does the customer understand the key behavior I want them to do?Does the customer understand how to get to that key behavior? Are the my instructions clear?