vwo - widerfunnel webinar | using learnings from failed a/b tests to create winning tests
TRANSCRIPT
© 2007-2017 WiderFunnel Marketing Inc.
Tweet this: @chrisgoward #Awesome
Learn From Failed A/B Tests to Create Winning Results
How to Turn Losses into Wins
© 2007-2017 WiderFunnel Marketing Inc.
Tweet this: @chrisgoward #Awesome
Before we get started. . .
• This presentation deck and recording will be available to you after the webinar
• Use the General Chat panel to ask questions or make comments in real time
• Stay to the end for the chance to win a free digital copy of You Should Test That! Type question here
© 2007-2017 WiderFunnel Marketing Inc.
Tweet to: @chrisgoward #Awesome
One of you will win the book today.
Get a free chapter at: YouShouldTestThat.com
© 2007-2017 WiderFunnel Marketing Inc.
Tweet this: @chrisgoward #Awesome
• Developed the LIFT Model®, PIE Framework, and Infinity Optimization Process™
• Keynotes at conferences and seminars around the world
• Author of You Should Test That!Get a free chapter at: YouShouldTestThat.com
Chris GowardFounder & CEO, WiderFunnel
© 2007-2017 WiderFunnel Marketing Inc.
Tweet to: @chrisgoward #Awesome
23.4% more orders for IBM Softlayer servers
© 2007-2017 WiderFunnel Marketing Inc.
Tweet to: @chrisgoward #Awesome
40.5% more ink subscriptions for HP Instant Ink
© 2007-2017 WiderFunnel Marketing Inc.
Tweet to: @chrisgoward #Awesome
115% more qualified leads for Magento Enterprise
© 2007-2017 WiderFunnel Marketing Inc.
Tweet to: @chrisgoward #Awesome
210% lead generation lift for Equity Trust
© 2007-2017 WiderFunnel Marketing Inc.
Tweet to: @chrisgoward #Awesome
$1 million per month additional revenue for BuildDirect.com
© 2007-2017 WiderFunnel Marketing Inc.
Tweet this: @chrisgoward #Awesome
But not all tests are winners, right?
© 2007-2017 WiderFunnel Marketing Inc.
Tweet to: @chrisgoward #Awesome
Result: Both variations lost.
Var A Var B
© 2007-2017 WiderFunnel Marketing Inc.
Tweet this: @chrisgoward #Awesome
But! The experimental design delivered valuable insights.
© 2007-2017 WiderFunnel Marketing Inc.
Tweet to: @chrisgoward #Awesome
Clarity: Shoppers want to refine their results by color.
..but, inconsistent use of “color” and “palette”creates cognitive load.
© 2007-2017 WiderFunnel Marketing Inc.
Tweet this: @chrisgoward #Awesome
There are no losing experiments......if you use the right experiment process.
© 2007-2017 WiderFunnel Marketing Inc.
Tweet to: @chrisgoward #Awesome
?
A B C
15.6% Lift 9.8% Lift23.6% Lift
© 2007-2017 WiderFunnel Marketing Inc.
Tweet this: @chrisgoward #Awesome
Great results aren’t random
The right process makes magic happen.
© 2007-2017 WiderFunnel Marketing Inc.
Tweet to: @chrisgoward #Awesome
Infinity Optimization Process
© 2007-2017 WiderFunnel Marketing Inc.
Tweet to: @chrisgoward #Awesome
ValidateDesign of Experiments is used to maximize the growth and insights results from experiments.
© 2007-2017 WiderFunnel Marketing Inc.
Tweet this: @chrisgoward #Awesome
Your challenge: Balance results and insights
© 2007-2017 WiderFunnel Marketing Inc.
Tweet this: @chrisgoward #Awesome
Unless you have infinite traffic...You can't isolate every variable - tests will take too long.
You can't only test one variation - you'll learn nothing.
You can find the ideal balance with Design of Experiments.
© 2007-2017 WiderFunnel Marketing Inc.
Tweet to: @chrisgoward #Awesome
The results and insights spectrum
Results Insights
A/B Multivariate
© 2007-2017 WiderFunnel Marketing Inc.
Tweet to: @chrisgoward #Awesome
The results and insights spectrum
Results Insights
A/B MultivariateFractional factorial
© 2007-2017 WiderFunnel Marketing Inc.
Tweet this: @chrisgoward #Awesome
It allows you to gain important insights.
© 2007-2017 WiderFunnel Marketing Inc.
Tweet this: @chrisgoward #Awesome
Use Factorial Testing to understand your customers’ psychology
© 2007-2017 WiderFunnel Marketing Inc.
Tweet to: @chrisgoward #Awesome
Infinity Optimization Process
© 2007-2017 WiderFunnel Marketing Inc.
Tweet to: @chrisgoward #Awesome
Explore
Web analytics – done right – can feed insights into user experience hypotheses.
© 2007-2017 WiderFunnel Marketing Inc.
Tweet to: @chrisgoward #Awesome
Conversion Rate: -2.23%Revenue per visitor: -2.95%
© 2007-2017 WiderFunnel Marketing Inc.
Tweet to: @chrisgoward #Awesome
Conversion Rate: -9.97%Average Donation: +1.57%Revenue per visitor: -8.55%
© 2007-2017 WiderFunnel Marketing Inc.
Tweet to: @chrisgoward #Awesome
Ikea effect
Consumers place a
disproportionately high value on
products they partially create.
Could interacting with the slider
create a sense of ownership of
the donation?
© 2007-2017 WiderFunnel Marketing Inc.
Tweet to: @chrisgoward #Awesome
Explore
Your visitors convert in their minds before they convert with their click.
© 2007-2017 WiderFunnel Marketing Inc.
Tweet to: @chrisgoward #Awesome
Loss Aversion
Studies have shown that losing
an amount is twice as
psychologically powerful as
gaining the same amount.
- Amos Tversky and Daniel Kahneman
© 2007-2017 WiderFunnel Marketing Inc.
Tweet to: @chrisgoward #Awesome
Conversion Rate: +2.04%Average Donation: +5.94%Revenue per visitor: +8.01%
© 2007-2017 WiderFunnel Marketing Inc.
Tweet to: @chrisgoward #Awesome
Consistency Principle
We like to keep consistent with
what we think, say and do, and will
change to ensure this is so.
- Cialdini, Influence.
© 2007-2017 WiderFunnel Marketing Inc.
Tweet to: @chrisgoward #Awesome
Conversion Rate: +1.97%Average donation value: +3.07%Revenue per visitor: +5.10%
© 2007-2017 WiderFunnel Marketing Inc.
Tweet to: @chrisgoward #Awesome
SelectedRegular donor
Selected Occasional
donor
Selected First-time
donor
Did not select an option
14% of visitors68% of donors
86% of visitors32% of donors
© 2007-2017 WiderFunnel Marketing Inc.
Tweet this: @chrisgoward #Awesome
Now, we’re building on these insights to deliver Personalized data-informed experiences.
© 2007-2017 WiderFunnel Marketing Inc.
Tweet this: @chrisgoward #Awesome
Q&A àOr email: [email protected]
© 2007-2017 WiderFunnel Marketing Inc.
Tweet this: @chrisgoward #Awesome
Want new ideas to spark your testing program?
Download the new graphical cheat-sheet with 17 new A/B test ideas:
WiderFunnel.com/17ideas
Email your questions to: [email protected]
© 2007-2017 WiderFunnel Marketing Inc.
Tweet this: @chrisgoward #Awesome
Thank You!You will receive a recording of the webinar
in the next few days.