po’ man’s guide to outrageous testing & optimization by ryan hutchings
TRANSCRIPT
#SMX #22b @hutchseo
$500k of consulting in 30 slides
Po’ Man’s Guide to Outrageous Testing & Optimization
#SMX #22b @hutchseo
Ryan Hutchings, Glamping.com Overlord
"Statistics are like a bikini. What they reveal
is suggestive, but what they conceal is vital." –Aaron Levenstein
#SMX #22b @hutchseo
It’s math – not marketing… ….math and educated guessing
CUT TO THE CHASE
#SMX #22b @hutchseo
CURRENT STATUS
§ Create a “test version” § Run for a couple of weeks § Compare to original version in analytics § Look at basic stats & conversion rates (if
possible)
#SMX #22b @hutchseo
WHY DOESN’T THAT WORK
#SMX #22b @hutchseo
METHODOLOGY
A/B Split Test • Internal system with server-
side splitting, custom variables
• Full manual reporting – spreadsheets and database reports
Larg
e te
st Search Results
Cart
Product
Test
ing
Landing Page Optimization • Third-party software • Internal analysis determine
what to test. Sm
all t
est
PPC and SEO landing pages
Headlines/CTA
Images
#SMX #22b @hutchseo
Step #1: DATA PREP
Landing Pages
Search Results
Property
Cart
Billing
Purchase
3.5%
1.7%
“rising tide” A/B (Large test)
Single page LPO (Small test)
40%
1.7%
Purchase
Know your conversions!
Site Conversion: 1.5%
#SMX #22b @hutchseo
Step #1: DATA PREP
Landing Pages
Search Results
Property
Lead
Sale
Purchase
4.5%
1.7%
“rising tide” A/B (Large test)
Single page LPO (Small test)
25%
3.0%
Site Conversion: 2.0%
#SMX #22b @hutchseo
STEP 2: METHODOLOGY
A/B Split Test • Internal system with server-
side splitting, custom variables
• Full manual reporting – spreadsheets and database reports
Larg
e te
st Search Results
Cart
Product
Test
ing
Landing Page Optimization • Third-party software • Internal analysis determine
what to test. Sm
all t
est
PPC and SEO landing pages
Headlines/CTA
Images
#SMX #22b @hutchseo
Landing Pages
Search Results
Property
Cart
Billing
Purchase
A/B (Large test)
Single page LPO (Small test)
Purchase
STEP 2: METHODOLOGY
#SMX #22b @hutchseo
www.meclabs.com
STEP 2: METHODOLOGY
#SMX #22b @hutchseo
STEP 3: LARGE & SMALL
A/B Split Test • Internal system with server-
side splitting, custom variables
• Full manual reporting – spreadsheets and database reports
Larg
e te
st Search Results
Cart
Product
Test
ing
Landing Page Optimization • Third-party software • Internal analysis determine
what to test. Sm
all t
est
PPC and SEO landing pages
Headlines/CTA
Images
#SMX #22b @hutchseo
SMALL TEST: MATH
47% Bounce • 53% stay
• 5,300 (visitors) • 50% CTR
• 2,650 (proceed) • 2% conversion
• 53 (conversions)
42% Bounce • 58% stay
• +500 (visitors) • 50% CTR
• +350 (proceed) • 2% conversion
• +7
7 more conversion • $150 / conv. • $1,050 sales • Small enough to
fall into a margin of error
10,000 Visitors
#SMX #22b @hutchseo
1. You optimize a bunch of pages and get wins everywhere 2. Then 3 months later your overall metrics haven’t really
changed – bosses start to think “They told me we should be seeing a 30% lift. I don’t see 30% more sales.”
3. Update LinkedIn profile and subscribe to daily emails from TheLadders and Glassdoor….that you never unsubscribe to…ever.
SMALL TEST: LIKELY OUTCOME
#SMX #22b @hutchseo
§ Tools: Optimizely (unbounce & visualwebsiteoptimizer) § Priority: Traffic + Performance (CTR then Bounce) § List all your pages, prioritize the list, always have 1-3
running § SMALL BEANS – in aggregate it’s good
SMALL TEST: DOWN AND DIRTY
#SMX #22b @hutchseo
SMALL TEST: PPC LPO
#SMX #22b @hutchseo
SMALL TEST: PPC LPO
427% Increase in CTR
#SMX #22b @hutchseo
SMALL TEST: PPC LPO
#SMX #22b @hutchseo
SMALL TEST: PPC LPO
58% Increase
#SMX #22b @hutchseo
LARGE TEST
A/B Split Test • Internal system with server-
side splitting, custom variables
• Full manual reporting – spreadsheets and database reports
Larg
e te
st Search Results
Cart
Product
Test
ing
Landing Page Optimization • Third-party software • Internal analysis determine
what to test. Sm
all t
est
PPC and SEO landing pages
Headlines/CTA
Images
#SMX #22b @hutchseo
q A/B testing platform q ROI template q Pre & post test calculator
LARGE TEST: Checklist
#SMX #22b @hutchseo
LARGE TEST: PLATFORM
Conditions Segment your users and/or their visits to single or multi-visit conditions.
inlineLeadForm1 mrSearchPage1 Notest simpleModalLeadForm simpleModalLeadForm4
control test
#SMX #22b @hutchseo
LARGE TEST: ROI TEMPLATE
#SMX #22b @hutchseo
§ http://www.kaushik.net/avinash/excellent-analytics-tip1-statistical-significance/
§ Download the template § Insert a “duration estimation” calculator to the template § Add 2 more tabs – daily tracking graph & screenshots
LARGE TEST: VALIDATION SHEET
#SMX #22b @hutchseo
LARGE TEST: VALIDATION SHEET
#SMX #22b @hutchseo
LARGE TEST: VALIDATION SHEET
#SMX #22b @hutchseo
LARGE TEST: VALIDATION SHEET
#SMX #22b @hutchseo
#SMX #22b @hutchseo
24% Increase
#SMX #22b @hutchseo
#SMX #22b @hutchseo
32% Increase
#SMX #22b @hutchseo
§ Just find someone purely organized (almost anal) and analytical
§ Skip the UI/UX experts (unless they meet the above requirements)
§ Take the time to create your templates right § Define “conversions” in reporting
TIPS
#SMX #22b @hutchseo
TIPS
50% +
40-‐50% 30-‐40%
>30% 60% +
50-‐60% 30-‐50%
>30%
Bounce Rates CTR Rates
#SMX #22b @hutchseo
TIPS Website Section Expected Improvement
Checkout/Payment Pages (Purchase) 15% - 25%
Product Pages (CTR) 10% - 20%
Search Results (CTR) 5% - 7%
Landing Page (Bounce or CTR) 20%
Lead Form Submissions 15% - 25%
