test for success: a guide to a/b testing on emails & landing pages
TRANSCRIPT
Test for Success A Guide to A/B Testing on Emails & Landing Pages
#abtesting
Our presenters
Jessica LangensandMarketing Programs Manager
Marketo
Allison SparrowMarketing Automation Manager
Optimizely
#abtesting
A few things first...
● Slides and recording emailed to you immediately after the webinar.
● Ask questions in the chat box & on Twitter! ○ Use #abtesting.
#abtesting
What you’ll learn
● Why should you A/B test?● Setting up your first (effective) A/B test● Testing Ideas and Success Stories
#abtesting
Why Should You Care About A/B tests?
#abtesting
#abtesting
#abtesting
There are 4.1 Billion email accounts in existence.Source: The Radicati Group
#abtesting
In 2013, SMBs voted that email marketing was #2 most important marketing channel.
#1, having a website.Source: eMarketer
#abtesting
2.3%{
#abtesting
A/B Testing is the process of showing different versions of a site to visitors, in order to see which one better achieves your goals.
#abtesting
An A/B test is...
#abtesting
#abtesting
One Page/Email
#abtesting
One Component
One Page/Email
#abtesting
One Page/Email
One Component
One Split Audience (50/50)
#abtesting
A B
#abtesting
A B
43.4% more pre-orders
#abtesting
Why should you care about A/B tests?
● Let the data do the work● Build internal alignment● Your website and email channels are invaluable
(make the most of it)● It’s easy, and cheap
#abtesting
What is an (effective) A/B test?
#abtesting
What is an (effective) A/B test?
● Identify your company’s goals● Choose where to test● Choose what to test● State your hypothesis● Track and report results
#abtesting
What are my company’s goals?
#abtesting
What are my company’s goals?
● Purchases/conversions● Revenue per visitor● Lead generation● Pageviews
#abtesting
Where to test?Welcome Page Product Page
#abtesting
Where to test?
● Welcome email● Home page● Product page● Contact Form page● 30 day expiration email● Download confirmation
#abtesting
What to test?
#abtesting
What to test?
● Headline● Subject Line● Value Proposition● CTA
#abtesting
Create a Hypothesis
“Visitors are getting distracted and feeling overwhelmed with too many competing elements on the page. Removing elements will reduce the clutter, focus user attention on article content and result in an increase in article pageviews.”
#abtesting
State a Hypothesis
● Identify a theory about the specific user experience
● Define the metrics for success● Define the quantifiable change in the key metric
#abtesting
Track and Report Your Results
#abtesting
How to Determine a Winner
#abtesting
#abtesting
Statistical Significance
Source: ZettaSphere.com
#abtesting
Statistical Significance
Source: ZettaSphere.com
#abtesting
Landing Pages
#abtesting
Significant?
Source: SplitTestCalculator.com
#abtesting
Testing Ideas
#abtesting
Format
• Font• Text Size
• Color
• Branding
• Images :)
#abtesting
Subject LinesA
B
#abtesting
A
B + 33% click through rate
Subject Lines
#abtesting
Subject Lines
● Vary a Replicable Element[New Report] 2014 Marketing Automation Vendor Comparison
vs.
[Download] 2014 Marketing Automation Vendor Comparison
● Complete Subject Line Test○ Test on a small group, wait 24 hours and send winner to
remainder of your list
#abtesting
“From” Line
“Scott Magri”
vs.
“Marketo Premium Content”
+ 4% opens +3% click-through
#abtesting
Button Placement
#abtesting
Button Placement
+6% CTR
#abtesting
Headers
#abtesting
Banners
#abtesting
Banners
10% CTR 5% CTR
#abtesting
Button Color
#abtesting
Button Color
+21% CTR
#abtesting
A/B testing isn’t about what you like. It’s about what your audience responds to.
#abtesting
#abtesting
Questions?Use Chat
#abtesting
Thank you!Recording is on the way!
#abtesting
Appendix - Resources for A/B tests
- statistical significance
- sample size calculator
- a/b test archive template
- variation calculator
- toolkit
#abtesting
No one has time to test everything...
Use Data to Determine what to test.