designing work
TRANSCRIPT
DesigningExperimentan
This is a brief guide to designing experiments
This is a brief guide to designing experimentsWARNIN
G!
Be sure to fill in all blanks before proceeding to the next step. Skipping this step will prevent you from getting the most out of this guide.
Now, with loss aversion in mind, let’s proceed to the next step...
This is a brief guide to designing experimentsWARNIN
G!
Be sure to fill in all blanks before proceeding to the next step. Skipping this step will prevent you from getting the most out of this guide.
Identify Key Behavior1
What is the key behavior you want your users to
do?
Identify Key Behavior1
e.g. sign up for workshop
Identify Key Behavior1
Key Behavior =Measured Variable (what you will measure to see the effect of your test)
?EXPERT TIPAsk around, does
everyone involved with your project agree that this is the most important behavior to measure?
Identify Behavioral Barriers 2
Identify Behavioral Barriers 2
What are the barriers preventing people from
doing the key behavior that you want them to do?
not enough time
read e-mail and forget about itput off until later
(list as many as you can think of)
Choose a Barrier3
A B C D
Choose a Barrier3
Choose the behavioral barrier (1) that you think contributes most to people not doing the key behavior, and explain it in the following format:
A B C D
Choose a Barrier3
The reason people are not currently partaking in
adopting negative keywords is because they put it off until later
key behavior
behavioral barrier
A B C D
4 Removing Barriers
Think about behavioral economic principles. What are some ways that you can remove this barrier? (list as many as you can*)
4 Removing Barriers
Precommitment: Put it on their calendar for them
In-your way instructions: Have a pop-up when they log into account
Urgency: Provide a deadline for the action to be completed
4 Removing Barriers
4 Removing Barriers*If you don’t
know principles, check out our workbooks
Choose the approach that you think is most likely to remove the barrier.
What is the behavioral economic principle that you are using?
5Choose a
Theory
By using urgency, we can increase the number of
people who adopt negative
keywords.
5Choose a
Theory
If we do this...then this will happen...
because...
If we add a sense of urgency, then people will be more likely to take action to add
negative keywords, because urgency gives people a clear timeline in which they must
complete the action in order to get the incentive
Write the Hypothesis
Creating Conditions(Control & Test)
How would you test this? Let’s start off with just two conditions (your most basic A/B test). Design your test below. What will your each group of participant see?
• Write out the exact text they will see?• Will you use images? If so, add the exact images they will see?
Control Test
Must have at least two conditions - control condition and test condition The control condition must be
exactly the same as the test condition, except for the ONE key difference you are testing.
Creating Conditions
Control Test
You must have a control
Make your ad more relevant. Adopt keywords and receive $50 in AdWords Credit
Make your ad more relevant. Adopt keywords in the next 7 days and receive $50 in AdWords Credit
Control Test
•The difference between the control and the test is the one thing you are testing
•Urgency: “In the next 7 days”•Restricting to only one difference between any two condition allows you to learn something from results.
•Imagine that you include 2 difference in the test (e.g., urgency and amount of incentive), and let’s imagine that the test works twice as well as the control. Why did it happen? You don’t know - it could
have been urgency it could have been the amount of the incentive, or a combination of both)
8Midterm Review
8Midterm Review
Do I have a control condition?
Do I have only one difference
between the two conditions?
Checklist
8Midterm Review
Given the information I provided,Does the customer understand the key behavior I want them to do?Does the customer understand how to get to that key behavior? Are the my instructions clear?